Knowledge poisoning
We all heard it a lot of times in our lives that “Ignorance is blessed”, when the proposal of not knowing something outweighs that of knowing about it, and the opposite of that: “The curse of knowledge”. Both sides have strong arguments going for them and everything boils down to a naive thought and easy to underestimate action of making a choice. A choice to either learn about a subject or to leave it on the table and just move on.
The bias of the previous, hard earned knowledge not giving into new concepts and ideas can be a serious roadblock in ones life and career. In this article I’ll try to focus on the prospect of knowledge poisoning in the computer science field, but aspects of that may be applied to other fields as well.
The hindrance of knowing too much
The human mind is very tricky and depending on a person, it has limited capacity. Some can store vast amounts of information and others simply can’t. As we gather knowledge overtime, we actively decide what will be put in our memory banks for later retrieval when needed. That means that some of our previously acquired knowledge is constantly being proven and tested for persistence when there is a need to store the new information in its place. Each time a new information is being received, the older, less frequently used (or remembered) is in terror of being switched with the new, fresh piece of that all valuable info that we steer for. Although this is, by all means, not a strict rule, it’s something that paints a realistic picture for most people. But what happens when the old information is constantly being reevaluated and again and again being the victor in the battle with new information streams? What happens to all that knowledge that lays on the battlefield?
In every fast growing industry field, knowledge is a constant stream and stubbornly sticking to the oldest information we know, can prove to be much more of a pot hole than a freshly laid tarmac on the information highway. If we frequently stick to our guns and ignore all the razzle-dazzle of new ideas and solutions, we set ourselves up for obsolescence. This is not a question of expertise and being a treasure trove of previously important knowledge about a subject, this is much more a tale of survival. Survival in the information age.
Not wanting everything to be painted with an IT brush, let’s look at the comic book industries between western markets and the eastern markers. Both very rich and decades worth of content at the disposal of those who want to explore it. But, every newcomer will at some point ask the question: “Where should I begin?”. Truly, with that much content and lore, where to begin?
In the western comic book industry, especially with some of the long-running series such as, for example, The Amazing Spider-Man, we can observe a trend of rebooting various titles every few years. The purpose of those frequent reboots or re-imaginings is to bring in new readers. If not for those reboots, newcomers might feel slightly overwhelmed when looking into the vast history of a comic book title. Reboots give a much needed jump-in point into the world without the burden of knowing everything that happened in the years and decades before, leaving the option to jump into those as well when the reader feels comfortable enough to explore the history of interest. In eastern manga industry, we can observe a quite different approach. While many of the available manga properties can have a concept of “sagas” that can be viewed as potential jump-in points into the universes they create, most of the properties are not split and in order for newcomers to understand everything they usually have to start at “issue 1” and progress towards the latest issues. Namely, manga authors and publishers rely on the strength of an intellectual property and pedigree to attract new readers.
We have seen two approaches in solving the issue of introducing a subject matter to new interested parties. Which one might be better? It is up for the readers to decide. Both have valid arguments for supporting their approach and equal cons of said approaches. Drawing a parallel with the software development industry, how easy, or better yet, how hard is it to jump into a new technology without prior knowledge?
In the early days of software development, needless to say, it was harder to come by any relevant information and the information highway was more of a dirt road with the occasional lonely traveler on it. Nowadays, so much information is available and access to it is perhaps easier than ever before. Now we come to the conundrum of the comic book industry, where to start? Do we start with the fundamentals or do we just go with online tutorials and learn as we go?
What happens to those who have accumulated so much domain and topic knowledge over the years when new software development languages, tools and libraries are introduced that are favored in the rising industry fields? Do they throw their hands in the air and plainly give up or do they try something else? On the one hand, they have the background knowledge and are well versed in the complicated processes that led to the creation of those new tools and libraries, but on the other hand, in result-oriented principles, that doesn’t add up too much. A junior developer that started fresh with the said languages and tools, can potentially have the upper hand in delivering a product to the market faster due to the clean-slate and fresh approach. What we have is the problem of having water pouring into a glass already full of water. Does one really need years-worth of computer science knowledge to follow a step-by-step tutorial that results in the same product?
Sticking to the old and proven ways has its time and place. Any respectful company would very cautiously approach new, unproven technologies. Experiment, yes, but going all-in could drive an unnecessary risk to a project. At one point the new technology will become enough saturated throughout the industry that it merits a thorough re-evaluation at which end those with great domain knowledge should be ready for a transition of knowledge to the new domain or risk falling behind. Oh, there’s another option, standing one’s ground on the technology standpoint. But exercise caution with this one. If not truly masterful in defending the older and proven principles, one could risk a lot.
Knowledge transition
Should you just go on and blindly adopt any new piece of technology and information that you might come across or whatever the latest buzzword is? Of course not. You should already have that cognitive maturity in place to decide when and where to adopt new knowledge in your particular field of interest.
Picking and choosing your battles is the mindset we are looking for. Like anything in life, we exchange our precious time for various goods and activities. Some are more closer to our understandings of “fair trade” and some fall in the “wasted time” section. The interesting thing to note here is that both piles are equal teachers and in some way complement each other. We as individuals can only learn from both and hope for better choices in the future. Software developers are constantly being put to the test with frequent updates and new shiny tools that come with lots of promises of improvements over the previous best thing ever. While being naturally curious, giving into these distractions can lead to shallow knowledge of almost anything.
How to decide what should stay and what knowledge should go? Hard to say that, because it’s very individualized. One thing that could help is asking oneself a simple question:” What value does learning this provide (to me)?”. This is a very powerful question. The first time we ask ourselves that question is the first time we step up from the sidelines and actively engage in our time allocation process. We look into the future and try to project the possible outcomes of that engagement. Not everything is precise of course, but the simple notion that we spent a millisecond of though on the subject, paves the way for clearer paths in front of us. One other way to decide what stays in our memory is just to give into the process. You might be wondering what that might be, but the simple answer is the normal way we preserve our knowledge most of the time and forget some other. The knowledge we are constantly testing by recollecting it time after time is more likely to stay in our permanent memory than the passing knowledge that we rarely use. I can speak from my own experiences in the software development field that topics that I was very comfortable with, lets say 10 years ago, I barely remember today. That simply means, that in the past decade, the use for that knowledge has been dwindling in favor of something else. At no point am I saying that that knowledge has been wasteful and under utilized, but that it just ran its course. There might be a point where that knowledge is going to be useful again and I’ll be sure to refresh it when the time comes. Until then, the new topics will sit comfortably where they are.
Like a river, knowledge needs to flow in order to emit a sense of freshness. Today, more than ever, we are being constantly overwhelmed with streams of information from each direction we look at, but its only up to us to know which stream of information we will subscribe to and dedicate our time and memory cells. Keep an open mind and don’t forget, you’re never starting from zero, you’re starting from experience.
Focusing on fresh new ideas
This is one subarea where it truly depends on the field of study and expertise someone is interested in. In general, we should be open to new suggestions, but reserved with modifying and changing existing ones.
If we approach new ideas with a sense that the more tools we have under the belt will make us somehow better and provide us with the sense of job security, we are just forming a thin veil of illusion over ourselves. Why an illusion? Because it is indeed helpful to have as much as possible knowledge on different topics, being competitive with that knowledge is a complete other thing and the job markets rarely go for candidates that can cover the bare minimum in one tool. On the other end of the scale, we have mastering of the trades. This is very promising, but in order to truly master something, time and effort have to follow. Individuals who are experts in a field are in high demand, but risk falling behind when markets change and something else is more in demand. Finding the right balance and juggling two or three different domain tools is probably more safe from a job security point of view.
Having an eye out for new topics and ideas, especially in our primary work domain has shown promising results. If one is to busy to look deeper into the new topics, just observing and listening can leave a door open to jump into it when and if the time comes. That time might never come, but at least we were informed and didn’t dedicate too much time with deeper exploration and on the chance that the topic has been picked up by the community, we can learn more just in time. Best way to look at it is like reading the morning news, not actively engaging, but at the same time being well informed.
No one wants to be the person in a team that constantly says “No” to everything without even giving it a shallow thought. At some point the questions will cease to come in and that’s the point where ones opining might no longer be considered. In software development, I would occasionally encourage colleagues to look for various alternatives to existing solutions. One reason is that I think we should limit the time for reinventing the wheel when there are already proven solutions for a problem available and the other reason is that we should always look a bit ahead and be prepared to quickly move to alternative solutions when and if the time comes for the existing implementations to be migrated from. Countless popular libraries lost the developer support behind them in a short period of time and developers must be quick to offer alternative solutions.
If we are looking for examples and proofs of how new ideas have changed existing landscapes, we can just look at how the industrial age provided the rise of the automation processes and making countless jobs obsolete. But not everyone has been impacted with those changes, some in those countless workers managed to pull themselves up, change gear and successfully familiarized themselves with the new ideas and tools. In fact, those who managed that incredible feet, were usually rewarded with higher salaries and better status overall. They simply managed to survive a tectonic shift in thought and knowledge. They had the capacity, energy and driving force to adopt to the new circumstances without letting the previous knowledge stand in the way, but guide.
While I constantly mention new languages, tools and libraries as the fresh ideas, those are not the only ones. Fresh ideas can be various things: thoughts, exercises, practices, verbal keys and so on. Countless of times we might haven been closed off to various ideas regarding team management practices, only to find out that they actually work very satisfactory. If working on a new project management technique, give it a month’s worth of testing before moving on. If trying out a new programming language, create an interesting new project with it. If you’re looking into a new food recipe, make the dish. Play with the idea of giving something from our interest domain a chance before making a decision that is tainted by previous experience.
Senior vs. Junior developers
Robert Martin wrote in detail (and talked on his presentations) about the expansion of new developers and the potential problem of knowledge transition from experienced developers to new developers. In short, there isn’t enough information exchange between the two.
While it’s not a total stretch to think that the early adopters of software development had it a bit easier than contemporary newcomers into the industry on the premise that there wasn’t as much information to absorb in the first place, at least compared to what a lets say, junior developer has to know on average in order to get a job, that can be a bit misleading. Yes, there’s so much to cover and know in order to stay afloat in today’s markets compared to the early years of software development, but there were extreme challenges back then too and there wasn’t any vast internet knowledge base to plug into for answers. The Internet had to be invented first. A surprising amount of core concepts in software development were coined decades earlier that when they became popular. The abstraction layer might have moved over the years and decades, but there were always hard challenges that needed expertise and creativity to overcome.
An interesting thing happens when a new employee joins a team or a project and is assigned either a mentor figure (usually a more experienced employee) or is the one providing mentorship. Depending on the level of expertise of the new member, a subtle shakedown might occur. There will be forces at play that will challenge each of the engaging parties. A new mind full of questions, ideas and approaches to problem solving and one with the possibility of providing answers with a great comfort level where it was before the questions came in. The key here being in recognizing potential useful scenarios and when to push forward and when to back up a bit and evaluate a newly formed situation. The mentor figure needs to be able to assert and control the eagerness of those seeking new knowledge. The process of information exchange between junior and senior developers is very rewarding for both sides. Junior developers receive valuable knowledge from experience and senior developers receive knowledge from learning. Both have foundations in the previous generation’s archives.
This can be an article all by itself and I might explore some of the problems in contemporary software development in it another time.
Upsides
Are there any caveats to sticking to older information banks? There are plenty. Like I’ve mentioned earlier we are living in the information age, any piece of information has its value. Perhaps the accumulated knowledge is of no particular use in one’s current area of expertise, but, for all we know, it can be useful in another. Keeping the old information alive is crucial to providing the gift of oversight and evolution of thought as it was in one particular time capsule in the years-spanning industry.The only way we can move forward is by building and learning on top of the previous collective knowledge.
Knowing how and where to showcase something that might seem obsolete in a modern environment can be tricky in the sense that not all of the participants speak that particular “language”. Backing every word up with examples and proof of concepts leaves other debating parties in a difficult position to contradict you. Arguments in the likes of something being “too slow” or “too hard” to implement, for example, can be contradicted with asking for measurements and concrete numbers backing those claims. The goal should always be to inform other participants and not to force of project something onto them. Good arguments are incredibly hard to contradict and if ignored, at least one can go with the sense of at least making an active effort to educate.
Being a good mentor figure and guide to others is a great and very rewarding skill in all by itself. A good mentor has the ability to make difficult things seem easy enough to comprehend. It is crucial to know how to communicate ones knowledge and adapt it to the listening audience.
Final thoughts
There’s a saying that trees grow deep roots in good times so that it can weather any storm in the future. Knowledge poisoning is just the previous gathered knowledge fighting back in an effort to survive the new waves that threaten its place in our memory banks as it rightfully should. What we can do is to implement sophisticated knowledge filtering mechanisms of our own that when vetted information passes through them, it will be welcomed by the old memory hosts with a dedicated seat at the table of knowledge.
Like with my previous articles and posts, these are just my thoughts on the subject. What is your understanding of the concept of knowledge poisoning? I like to hear your thoughts.
