Dark Matter Developers: The Unseen 99%
In astronomy and cosmology, dark matter is a currently-undetermined type of matter hypothesized to account for a large part of the mass of the universe, but which neither emits nor scatters light or other electromagnetic radiation, and so cannot be directly seen with telescopes. - Wikipedia on Dark Matter
You can't see dark matter, but we're pretty sure it's there. Not only is it there, but it's MOST of what's there. We know it and we can't see it. It never shows up.
My coworker Damian Edwards and I hypothesize that there is another kind of developer than the ones we meet all the time. We call them Dark Matter Developers. They don't read a lot of blogs, they never write blogs, they don't go to user groups, they don't tweet or facebook, and you don't often see them at large conferences. Where are these dark matter developers online?
Part of this is the web's fault. The web insists on moving things forward at an rate that makes people feel unable to keep up. I mean, Google Chrome has upped two version numbers just in the last 3 paragraphs of this blog post. Microsoft probably created a new API and deprecated an old one just while I was typing this sentence.
Lots of technologies don't iterate at this speed, nor should they. Embedded developers are still doing their thing in C and C++. Both are deeply mature and well understood languages that don't require a lot of churn or panic on the social networks.
Where are the dark matter developers? Probably getting work done. Maybe using ASP.NET 1.1 at a local municipality or small office. Maybe working at a bottling plant in Mexico in VB6. Perhaps they are writing PHP calendar applications at a large chip manufacturer.*
Personally, as one of the loud-online-pushing-things-forward 1%, I might think I need to find these Dark Matter Developers and explain to them how they need to get online! Join the community! Get a blog, start changing stuff, mix it up! But, as my friend Brad Wilson points out, those dark matter 99% have a lot to teach us about GETTING STUFF DONE.
They use mature products that are well-known, well-tested and well-understood. They aren't chasing the latest beta or pushing any limits, they are just producing. (Or they are just totally chilling and punching out at 5:01pm, but I like to think they are producing.) Point is, we need to find a balance between those of us online yelling and tweeting and pushing towards the Next Big Thing and those that are unseen and patient and focused on the business problem at hand.
I like working on new stuff and trying to new ways to solve old (and new) problems but one of the reasons I do like working on the web is that it's coming to a place of maturity, believe it or not. I feel like I can count on angle brackets and curly braces. I can count on IL and bytecode. These are the reliable and open building blocks that we will use to build on the web for the next decade or three.
While some days I create new things with cutting edge technology and revel in the latest Beta or Daily Build and push the limits with an untested specification, other days I take to remember the Dark Matter Developers. I remind my team of them. They are out there, they are quiet, but they are using our stuff to get work done. No amount of Twitter Polls or Facebook Likes or even Page Views will adequately speak for them.
The Dark Matter Developer will never read this blog post because they are getting work done using tech from ten years ago and that's totally OK. I know they are there and I will continue to support them in their work.
* These people and companies all exist, I've met them and spoken to them at length.
UPDATE: I found a blog post from John Cook that mentioned Kate Gregory who also used this term "Dark Matter" to refer to programmers. It seems it's an apt analogy!
About Scott
Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.
About Newsletter
Most developers deliver real business value, day in and out, using chaotic or highly traditional (read: waterfall) processes where their success isn't measured by how cutely they can integrate the latest IoC Container but rather "is that report handling the latest acquisition correctly?".
Some times, it's really worth pushing your company to try out the latest agile thinking or web API. But it's also realy easy to underestimate how you could kill their productivity because they're not in a position to take advantage of the advantages and you just moved their cheese in a big way.
It's a bit like taking someone's minivan away and handing them a Lotus Elise. Yep, it's an amazingly fast car. But on a commute to and from work each day very few people are in a position to take advantage of it. On the other hand, they'll definitely notice the spartan interior, simple HVAC, and inability to see around the other minivans filling up the road.
But...
I feel I was still more productive when it came to business functionality delivered when working in the AS/400 environment as a dark matter developer. Using Synon/2E and RPG. Stable, robust, proven, ancient.
Actually nobody's sure that there is dark matter at all. Not even those vocal "scientists". Dark matter (and some other exotic entities) are just mathematical constructs invented so that the Big Bang theory stands.
But I agree and like the article about unseen developers that get things done.
Similarly, I've been fascinated by Ayende's posts on architecture where he eschews repositories, "traditional" dependency injection, and everything else we've been told is absolutely necessary in favor of an "enterprise architecture" featuring a grand total of 8 things. It's refreshing to see someone as prominent as ayende promoting a style that is simple because he has eliminated as much overhead as possible. It's wonderful to be able to read and understand his code, which I can pick up even just reading it inline in his blog posts.
Anyway, I like the kind of simplicity that happens when your framework/architecture is small enough to be understood.
It's late.
http://www.ted.com/talks/susan_cain_the_power_of_introverts.html
There is a vast range of developers (as in how outspoken they are online). I wonder if there is a relation between being extrovert/introvert and the online-presence that one creates. I would assume there is a strong correlation. Extreme extrovert people would tend to be on twitter, facebook, linkedin even google+, their blog, github, ... and be present with a lot of 'noise'. On the other end of the spectrum I suspect extreme introvert people to bare register online. I have met some people like that and they do not even know there is a site like stackoverflow for example.
As in the TED video, the best is to acknowledge ones own position (extrovert vs introvert) and acknowledge that either 'side' has their benefits. Together we are stronger.
Daniel, i think dark matter hasn't got much todo with the big bang theory, it is a construct to explain the discrepancy in the gravitational structure between eg a galaxy with only its luminous mass and what is actually observed. Short of changing the theory of gravitation, dark matter seems a not so bad hypothesis.
First type, developers that find a (good) job that they work long hours, and produce software for a specific app/suite. They are too busy working on it while being rewarded for it, so they are too tired to "look around" and don't really care. They like their job, but do not have an "actual need" to learn about new stuff.
Second type, developers, that have become devs, not because they actually wanted to, but rather because they saw it as one more career option, in the same way that they would become finance analysts, or marketing execs. Once they get a job that pays the bills, they are content with it and do not care what goes on in the software world. As Craig mentioned, they go home and watch tv...
Despite this, to any conference and presentation on Microsoft and .NET technologies, you won't see more than 60 people (among which probably 40% are students), no matter if the event is organized by the local .NET groups or Microsoft itself. Even among those who come to a presentation, many will come only once out of curiosity, then never again.
In the online forums it's the same - no more than (the same) 10 - 20 people are active.
As far as I can tell, countless developers seldom read a blog, never user twitter, and very, very few have an active blog of their own.
For many, the latest and greatest technologies are great, but they don't matter as long as they have to develop daily, 8 hours/day, a huge WinForms application started only 3 years ago.
I agree that there are many dark matter developers out there, but I just wanted to give, perhaps, a different take on some of them. I consider myself a dark matter developer, but not because I don't read blogs or attend (many) conferences.
I think of myself as an above average developer (purely based on the number of tests I've taken for interviews. Can't say I'm a super developer based on the test/criteria you put in one of your blogs), who has a voracious appetite to learn everything new. To consolidate what I've learnt, I need to practice, and naturally, work time is not conducive to this, as I'm there to work, not play (unless you're either very fortunate that you have little work to do, or you have an accommodating boss who gives you R&D time), but, because like a lot of people, I have a 3 hour round trip commute every day, which leaves you little time to practice. To be honest, although I've very grateful for your fascinating tidbits, I don't really understand how have the time to play & blog, when you appear to run a very busy department. Hat tip to you and everyone else who can manage it. In the end, you have to realise your limitations and just do your best.
As for blogging, why would I, just to re-gurgitate stuff which has been said a million times by people far better placed than I. And not every one is a natural writer, you should be thankful for the gift you have. As for Twitter, for me that's just noise, and there's too much sifting the wheat from the chaff; can't be bothered. Anyway, RSS is excellent. Facebook? Don't get me started; assuming there's 800 million FB users, I'm more than happy being one of the ~5.2 billion not on FB, having my privacy eroded.
There was a time when I tried to keep up with a lot of what people were saying, but feeling constantly frustrated that there wasn't enough time in the day to read everything. In the end, I just had to choose to follow people who chime best with me and except that you're going to miss a good deal of information.
There, that's off my chest. But do keep up with your excellent R&D and blogging.
In Russia, there are even more "dark matter" coders, because IT blogosphere is very elitist and topics discussed there are mostly "rocket science" like Haskell (or something more complex like Agda or Coq).
First, I would like to be more like them. Every time I sit down to work on a side project at home I find myself updating all my libraries to the latest and greatest and doing very little actual productive work.
Secondly though, some of these dark matter devs are writing code that is downright horrid, and I've had to debug and fix some of it. For some reason there is a strong correlation with VBA, though it can be written well. There is a reason for new fangled patterns and technologies, and usually they help to make maintainability and readability better.
(I have a sneaky suspicion that web things will stabilise soon on one-page sites using web api but with simple fallback to multipage, a bit like project silk. Looking forward to seeing what you do with the single-page stuff. EF Code first or the fluent NHibernate equivalent seem pretty stable already.) That wasn't a third point at all.
I can't compete with an AS400 RPG programmer in productivity.
And the users are more productive in a greenscreen app with function keys than a fancy WPF app.
But what I "hate" now is those programmers doing OVER designing. X patterns in a stupid CRUD application to do 2 steps. A composite pattern, with state and with a decorator and ...
Just keep it KISS and clean.
As I sit here writing code in VS2005 and SQLServer 2000, it's nice to know that I'm not forgotten.
But maybe they would care if they understood that it could help them have new features in half the time/cost (after the initial pain of upgrading). Though again the risk is that you endlessly spend money updating and never actually give them any features.
I think that this same people write code that works, but dont write code that continue to work.
Write something that works now is very easy, but something that is maintainable is when we see the difference between good and bad programmers
We tried that and unfortunately a cost to this quarter's budget to save on the next quarter's budget is a concept that a lot of our customers have a problem with.
Our technical debt is through the roof, but our customer's aren't prepared to pay for us to do the housekeeping (as they pay us by the hour they keep tight control of how we spend our time).
Belbin says you need various types of developer to make a high performing team. But stagnating developers could add risk to a team/project. They require more training, are less likely to get things right and often require carrying forward.
The people I work with have done really cool things that probably no one knows about:
* Built our own app store for Android/iOS for employees
* Built single-sign-on for Android/iOS using our AD infrastructure, both native and hybrid apps
* Lots of SAP to .NET magic for front-ends
* Custom building of iOS/Android applications using SSH/web services from our build tool (Windows) and Mac Mini (I did a lot of the SSH stuff)
There's just a lot of stuff that these guys & gals do that stays inside and no one knows about.
Thing is, as a producing programmer I've been in a learning phase for roughly 80% of my career, which started with a beta version of Windows 1.0. And as a producing programmer, I need to learn a lot of non-programming stuff at the same time. NerdDinner, AdventureWorks and the like are good teaching tools, but they don't begin to capture the morass of competing requirements and demands that real-world developers struggle with. Finally if I'm successful on a given project, I usually have to stick with it for a few years before I can move on to the latest and greatest.
So I'm perpetually behind the curve. I once heard a measure of mastery that went like this: if you write down everything you know on a subject, how big would the book be? At one point, I could have written a pretty good book on C++, but by the time I reached that point, there was no need for another book on C++. Currently, I'm neck-deep in technologies that I couldn't produce a decent pamphlet on, as has usually been the case.
Which is why I keep reading. By the way, Scott, your stuff is usually pretty decent. Thanks, and keep it up.
Just to add to what Robert S. said -- I am also an active .NET developer who seeks out the latest libraries, patterns, and techniques. I try to keep on top of what Microsoft and others throw out each year. I try to listen to .NET Rocks, Hanselminutes, and anything else I can to make sure I'm aware of what is out there to help me with the applications I build.
But I don't have a Facebook or Twitter account. I don't have a blog and I have zero online presence. And that is by choice. I think there are a lot more of us than you think. For me, personally, I do not have time to engage with social networks and I have even less desire to do so than I have time. I'm not an introvert by any means -- I just have no desire to become "Internet famous" or be able to Google myself and see my footprints everywhere.
I hope that doesn't sound like a slam on you. I wasn't being specific. You do what you do because you're great at it and it is something you feel compelled to do. I would rather spend my free time with my family, playing the guitar or writing applications for side businesses that I have. So maybe we need a category for people like me. We're like pulsars I suppose. So now it's time for my light to go off again...
I get a little annoyed by devs who are always shining the damn thing all over the place, they remind me of crows who are after shiny baubles all the time.
Just get stuff done. Play at home, and then when you've discovered real value, bring it. Until then, please refrain from littering those damn baubles all over the place. I hate seeing good opportunities turn into someone's science project. It's actually a little irresponsible at times.
Maybe many of them will. Just because they don't comment doesn't mean they don't read either. I know many of my colleagues follow blogs, but they don't comment them. I myself often read your blog (and a few others) but never (or extremely rarely) write any comments.
Must be my personality :)
- Experts in technology are often introverted, and will therefore usually not participate in the usual social mechanisms. (e.g. I can't even relate to or understand why anyone would want an iPhone, Android, or WinPhone.)
- They are often deeply knowledgeable about a range of tech and non-tech topics, but it takes years to gain this knowledge. Unless you've spent 10+ hours a day for 5+ years working exclusively with some tech, then you are NOT an expert!
- Experts don't write books. They spend 5 years designing and implementing a product, then come up for air, and find that those in power have decided to completely change everything without even consulting them about what was right/wrong with the previous technology.
- Being an expert is not a good feeling. You know you're an expert when your are stuck trying to figure out some horrendously difficult problem, and you realize that there is no one who can help, because you know a particular area of technology as well as anyone. If you can't figure it out, then there is no wise guru or mentor to come to the rescue. There are no smug contented experts.
- Experts strive for simplicity, but a problem with experts is that the smarter they are the more inscrutable their thinking is to lesser mortals. Microsoft probably has many experts, but I suspect they're smarter than me, because I don't find many of their designs simple enough.
- I prefer humble experts. The proper attitude is "Oh no, I'm smarter than everyone else. I hope nobody notices.", and "Although I can tell, unfortunately, that I seem to understand things more deeply than anyone else, I also notice that I'm still an idiot who can't understand anything as deeply as I should.".
Disclaimer -- I'm not actually an expert in most things. Most of my career has been designing and implementing software that takes 6 months to 2 years from start to finish. This is not enough time to reach truly deep expertise. But I *have* spent the last 5 years working on a technology that I hate (Java/Swing), and I'm almost starting to feel like an expert with that. I've been starting to suspect that the grass is greener on the VB/WPF side of the fence, but MS will probably kill off that technology before I have time to find out.
Microsoft (and others) need to do a better job of coaxing information out of the actual experts in their technology, because they consistently make avoidable mistakes. I've been playing with VS11 beta lately and noticed a number of them.
For example, removing the macro feature is a classic case of misunderstanding the data. Although few use the current feature, this doesn't mean that a macro feature is unnecessary or unwanted.
Here's the real history...
Visual Studio had a decent macro facility that could be used to solve real problems. The key feature was that it was easy to use, and didn't require loads of time to understand a complicated API. This macro facility was discarded in VS7 in favor of a "more powerful" one that was orders of magnitude harder to use. Now in 2012 that decision has finally caused the useful feature to be killed off completely.
The speed at which the technology changes is, I think, key to moving forwards. As .Net has evolved, so have the various methodologies and associated technologies, and I struggle to find recommendations and guidelines that I can trust, particularly as there always appears to be some infighting involved. So I've hit a kind of analysis paralysis where I don't dare make a move in case it's in the wrong direction. Taking the Entity Framework as an example: firstly there was a big buzz about ORMs, then the Entity Framework appeared and was rubbished in some quarters of the internet as not being a proper ORM, but that has now progressed through various versions so I find myself asking such questions as: Is it still rubbish? Was it ever rubbish and the people criticising it were wrong? Who can I trust? Another example: recently there's been a series of blog posts where Northwind Starter Kit from the "Microsoft .Net: Architecting Applications for the Enterprise" book has been heavily criticised. This is now quite an old book, so is it still worth reading, or is it out of date? Was it ever worth reading, given the criticism that the NSK is getting? As an outsider reading the arguments going on, I am unable to decide who's right. Are these pragmatic arguments, or are people just arguing for some academic reasons that have no bearing to the real world? It's a question I can't answer without being fully able to understand the problems that these solutions do or don't address.
But respectfully, I don't agree with the dinosaur-like characterization of those (like me) who don't blog, tweet, facebook, or indulge in other online forms of digital flatulence. (True, it's not all like that, but the other side of this equation is that far less than 1% of online content of this nature is worth the bandwidth that each byte of it occupies. Indeed, we need better tools to sort the garbage from the gold, as it were.)
I don't believe that I fit the archaic mould you describe. I do web development work for large corporations (indeed, my work history shares an employer with yours!), often using bleeding-edge tools and techniques, but most often I need to balance the use of these tools with technical risk. .Net 4? Still a bit early for the majority of corporate environments, IMHO. I have no doubt that will change very soon, but there's simply no time for me to mess with that stuff when right now there are tangible things to deliver that need to instantly integrate with exiting platforms. Software vendors are so adept at making everyone feel that what they are currently developing is instantly obsolete. The process breeds an insidious wave of attention deficit disorder. We can barely begin adoption of a new technology without being bombarded by pundits who tell us that our plans are already out of step with the times.
And while I celebrate the ability of many of the current social media to bring people together, retain links with your deep, dark past, or whatever you wish, I fervently believe that there is a difference between cyber-reality and, well, reality. In other words - not to put too fine a point on it - some of us simply have a life. And for many, these social tools turn them into, well, social tools. (Sorry, a cheap joke but I couldn't resist.) How many times do you go out to a restaurant and see couples sitting together, tweeting and texting others on their mobile devices? They have no sense of the here and now.
IAC, thanks for a provocative post. Keep 'em coming.
and read books to them and tuck them in every night. There are non-social folks that would be hard-pressed to say the same thing, no?
There's no judgement express or implied in my post, just simply an acknowledgement of the existance of different kinds of folks. Your thoughts?
The problem is that computers started by a small group of passionate people who wanted to make a better world for others and they spent best years of their lives at a young age building software when there was no tooling and libraries to assist them. At that time, programming wasn't a prestigious and well-known job with a very high income even more than man other specialized jobs.
Thank to their efforts, we're living at a time when computers are very common, and all the businesses need computer-related tasks which require programmers to write code. On the other hand, many companies (especially Microsoft) have made the task of programming super-easy for many common cases. Therefore, a person without much of a background in principles can write many of the data-driven applications using these tools without caring much about the community and practices. Such a person is not passionate about what he's doing, so as long as he's *just done* with his responsibilities to get user's data and display them back to him/her, he's satisfied, and can go home have a life like others.
To be quite honest with you, this is a very frustrating point for me. Not only in the context of the programming jobs these people have flooded the community with a negative fingerprint making it harder for passionate people to breath, but also you see such motives among the youth for entering the computer-related majors at universities. Everyday, I witness many junior and senior college students who entered this field for money/career reasons without having any passion. Upon graduation, they're unable to write simple programs.
Just my two cents. Thanks for your post :-)
There is a constant business demand for VBA and budget constraints (almost 0) mean that C# 4 and mysql fill in the remaining gaps.
My note of advice to Microsoft : big tech stacks are good for big projects and teams. A small stack is equally useful for developers churning out LOB apps that may only get used by 10 people - this is why winforms works so well still.
Then there are some areas where you can't afford to be left behind. I think web development is one of them. And it's not just about using the latest and greatest server-side technologies. It's also about keeping up with browser trends, mobile trends, how your audience changes over time, etc, etc.
Something that has changed drastically over the past decade is web developers' understanding of web performance (at least for those of us paying attention). I believe you interviewed Steve Souders at one point. He's been instrumental in getting the word out. And his YSlow tool is really cool too. Any web developer who doesn't follow blogs or really try to keep up with things could easily miss out on little gems like this.
I think it's natural, the way the dev universe is or should I say that's what makes it beautiful. Some's shine and some help shine (dark matter dev's).
They use the internet to find answers when they need and thats it. But they usually are just not that much online, not only do they not follow technical blogs, they don't follow blogs at all, or use twitter at all.
Not all dark matter employees are fulfilled by a paycheck and some "code that works"; many of us do have aspirations and progressive desires, and are embarrassed or dissatisfied about working in a shop spackled together by WinForms, SHTML includes, and PowerBuilder lines. They want to get out of their glacial-speed public service positions, but feel light years behind the nerds that tweet about their MVC4 Web API. These things contribute to the "<a href=http://www.hanselman.com/blog/ImAPhonyAreYou.aspx">phony vibe</a>" you brought to light so well back in August, making it hard for some to grow-a-pair and take risks.
Add to this lack of fulfillment/confidence the speed at which the IT world moves, and we begin to spend the majority of our career trying to stay on top of our careers. This scenario is arguably the worst of both worlds, as you're not the guru everyone wants at their startup, and you're no longer the trusted productivity guy because you've been putting the smack down on so many tutorials and code exercises.
We hear from our mentors and heroes online, "build some Stack Overflow rep", "get a blog rolling and show people what you got." Some lurkers give it a shot... then are quickly discouraged when their SO questions is down-voted for being too stupid. Not a very welcoming community.
Qerim said it well, the pundits of modern technology and programming theory (yourself excluded) often come off as "crows after shiny baubles"; or better yet, bluejays that just want people to know they are there and get some attention. Knowing well, not many of us are the Scott Guthries, Linus Torvalds, or John Resigs of the world - do we really need more chirpy birds?
Then there are those who say "the problem is when dark matter developers get promoted via the Peter Principle", or "they're fine until they get promoted to enterprise architect", or "they write code that works but doesn't continue to work".
I apologize for ranting, but I there is a point under it all and brings up questions for discussion:
1. How does a developer minimize and/or deal with being stretched over the log? (i.e. losing his/her productivity whilst investing time and money in training efforts)
2. How does a developer participate without being another chirpy bird?
3. How should a developer without much of an online presence go about networking with the "guys in the know"?
Thank you Scott, for another informative and provocative post.
I also observe that those who are "dark" and don't keep up are solid producers, but I also don't believe they push technology in a way that makes it more efficient. Work for any huge bank or insurance company and observe the mounds of COBOL programmers still necessary.
What I think I'm getting at here is that both ends of this spectrum are, in my opinion, not working toward the "right" goal. That goal is to leverage technology to do "stuff" in a faster, more efficient way. Less work, more result. That's the world I dream of. I love the tech, I blog (occasionally), I do the open source thing, etc., but when I'm done working, I want to hit the hot tub or plan my next trip to Hawaii, or just wrestle my kid... because I used tech to make that time.
Priceless. It's also more than a bit annoying. I have a computer at home that sits in my attic and does... stuff. I remote into it every couple weeks or so for some reason or other. It's always asking me to upgrade Firefox, install windows updates, update Flash, blah blah... I mean, can't it just be happy to hang out and get old?
But really, as someone that tries to keep up, I've realized how completely futile it is. You can't possibly be an expert on the cutting edge, even in a narrow area of focus, and get your work done. At least I haven't figured out how.
I accept that there will be some balance. I will not always be as cutting-edge as I would like. I will spend effort learning new, better things when I am dissatisfied with something old. But if I am using something that works and is not overly burdensome to use, I will try not to care if it's old, and I will promise that I won't replace something functional with something newer and slicker just because it's newer and slicker.
Because I don't have time.
Actually that's a bald faced lie, I waste tons of time learning about new things even when the old one is good enough. But I try not to do that when there's work to get done.
I don't know where I belong there. I read blogs, I use FB socially when I have time. The rest, I don't have time for. In the last year, I have started new projects using Silverlight, Entity Framework, and MVC. But I still support client apps I wrote in ASP.NET 2.0 and, God help me, VB 6.0 (it is at least on the way out - replaced by Silverlight, another soon-to-be has-been). My business is about "get the job done" but also "support and enhance the job done". Most of the time, that doesn't include changing frameworks and retooling the app.
I don't complain about any of this. I do my best to stay fresh because each new technology makes my job easier and keeps my job fun. I love this field.
Thank you, Scott, for the nod in my direction. And thank you for helping to push the technology forward to the next new thing. I think the world needs the 1% and the 99%. :)
Look forward to meeting you at the Dallas Day of Dot Net
If anything, the essence of what I'm saying is that people like myself just can't keep pace with everything and have a life as well. I too have a wife and children, and my guiding principle is that "I work to live; I don't live to work." (Although sometimes this isn't much in evidence given the amount of time I spend at the keyboard.) IAC for me, as much as technology is what I work at, I really don't feel that it defines me per se.
And then, of course, we all need to (and should) make our own value judgments on the impact on our lives of technology...hence my "social tools" quip. But I wasn't targeting anyone here, just projecting how I would look upon myself hypothetically. That probably says more about me than anything.
Sorry for any inferences!
Some days I'm totally stoked and crank away learning, other days I feel like I'm on lap one as people are approaching the finish line.
I really liked what you had to say - especially the last thought about measuring the impact of technology on our lives and our responsibility to make our own judgements accordingly. A broad statement, yes, but I like your point.
Take care.
Website I will write in the server's local scripting language but I prefer PHP.
Games (for PC) I used to write in QB (tho I never published a QB one) but wrote my last one(s) in VB6.0PE
Maybe I am one of your "dark programmer/developers"
BUT: I read (your) blogs and have at least some inkling of the latest developments in (more than just) the programming world so...
Maybe I am not :p
Actually nobody's sure that there is dark matter at all. Not even those vocal "scientists". Dark matter (and some other exotic entities) are just mathematical constructs invented so that the Big Bang theory stands.
That just isn't true. There are quite a few lines of evidence for the presence of dark matter. It is revealed by some gravitational lenses and the speed the sun orbits the milky way. Also .... most of the missing stuff is dark energy, not dark matter.
Just because people don't post blogs doesn't mean they are not enthusiastic -- they are just getting on with it. Developing is much more rewarding that prattling on about it ... so.... click
Personally I have found it to be a really hard time balancing the sheer amount of volume of information Ive been discovering against actual time I have to consume everything. Then I get into a mental funk where I hate myself for "getting on the bus" so late (well more so wasting the last 4+ years NOT consuming this information as much as I am now).
Another part of the problem is (many) years ago I was an avid classic asp developer, and I had such a hard time adapting to asp.Net in general - a part of this was my job, which didn't really make it any easier, and the other was just the abrupt change of directions .net made versus classic asp. Struggling with that for years before I really got comfortable with 2.0, only to realize that 3.5 was out, 4.0 in beta, and by the time I started messing with 4.0, 4.5 was in beta!!
That being said, I truly believe everything happens for a reason, and I have been having a blast learning all of this new and exciting technology. Its just a constant battle to NOT let oneself get completely overwhelmed and just walk away.
As always, thank you for running such a great blog and being a part of the amazing Microsoft community of developers!
Oh, and thank GOD for Pluralsight!!!!
I still read blogs, follow trends, learn new languages etc., but I have been in the business long enough to be able to separate the wheat from the chaff; taking on board the good bits; and applying it in legacy environments.
It should not be an 'either / or'; but a 'best of both'.
My coworker Damian Edwards and I hypothesize that there is another kind of developer than the ones we meet all the time. We call them Dark Matter Developers. They don't read a lot of blogs, they never write blogs, they don't go to user groups, they don't tweet or facebook, and you don't often see them at large conferences. Where are these dark matter developers online?
**
Not theory or hypothesis - it's fact! These are also known as nomad developers. I've lost count of the times I've polled a group of developers to ask them what blogs they read, magazines they subscribe to, etc, etc. The VAST majority of developers are not engaged. For most that program (I don't say "our business" because our business is one of passion and profession) are not engaged. Whether they get more stuff done or not - I don't know. While there may be some correlation, I don't believe there is any sort of causation here.
The simple fact is this - there are a good number of folks who sling code as a job and they don't give a lick about staying current, learning new things, sharpening the saw, etc. For us - that is completely foreign. There are also a fair number of folks who stick to themselves - but do keep current, look to learn new things, etc.
As for me, as I get older and have been in the business a while, I feel the need more than ever to try and stay at least a little behind the bleeding edge.
Without going over old ground the good old silverlight debate is a perfect example.
"persuade you company to send you yoi Tech-ed / Mix come and meet the industry experts etc"
2 years on Where are they now ??? raving about Jquery / HTML5 / Node.js / Working in Disney / playing with Netduino !
There is a striking parallel here with the behaviour of software evangelists and politicians. Whatever is the next best thing i should be talking about this week.
The rest of the world often has a job to do and learning newer more complicated stuff is a full time job in itself.
A poor but relevant metaphor might be if you and a number of other well known evangelists were to form an orchestra. Each pick an instrument and learn it to a standard you can perfrom in public then when you are all set to go on stage a week before you do just swap instruments !!!
don't get me wrong I love technoology and the pace of change but those of use that invest our time in trying to stay current with everything day in day out might want to step back and reflect on somehting mrs M said the other day.
"You might enjoy working on that computer most of the time, but that kinect and your comapny won't miss you in 5 years from now when it's moved on and obsolete, but your kids will be grown up and gone and what in the end have you actually achieved?"
food for thought !
B.....
@Json Krol - +1 for the Pluralsight plug; I'm really liking their post-course "assessments" - though I'm not sure how much a considering employer would respect them. Tekpub and Peepcode are awesome too if you haven't checked them out.
One page web, sounds like Classic ASP. MVC3 EF creates multiple pages for you, just right click and create... Guess I could go in and make some of them partials...
That is the kind of thinking those of us that belong to the DMD Army do daily (at lunch when I am reading Hanselman or Gu, or xyz).
You characterise yourself, and others like you, as 'pushing things forward', 'pushing the limits', 'pushing towards the next big thing'.
You claim to like 'working on new stuff and trying new ways to solve problems'.
You then go on to characterise others as being the type of developers who like 'chilling and punching out at 5.01pm', 'using tech from ten years ago' and refer to ASP,NET 1.1, VB6 and PHP.
I have worked in the games industry and I think that those I worked with would consider their efforts to push hardware to its absolute limit as far more cutting edge than tooling around with the latest web api or scripting language. Just a guess.
I would consider myself dark matter developer by your basic definition. No blog, don't tweet. no facebook account. Outside of my immediate environment you wouldn't know I exist. Though I do appear to read blog posts...
I currently work with a company which, among other things, explores massively parallel architectures (e.g. gpu programming targeting systems with 16k+ gpu cores), vertualisation, interpreters, JIT compilers etc. Some of which forms the basis of the toys you might play with, possibly even the ones your latest greatest tech is utilizing. Technology that might be categorized as bleeding edge, but no one at the company writes a blog, or tweets, or posts facebook status updates, or even attends large conferences (well, maybe the biz dev guys, they have my sympathy).
My language of choice is C with some assembly thrown in and occasionally C++ when pushed. Archaic stuff, I must be close to fossilisation.
Guess some people just think that its not what you use but what you do that defines being progressive, and that the latest new thing isn't necessarily synonymous with progress. It may also be true that those that aren't shouting may not be drudging either.
It appears that two is an impossible number, even when it comes to characterising developers. Disappointing article.
I don't at all categorize Scott or Atwood or Joel Spolsky this way; it's really more a function of the many commenters who seem often to do the equivalent of deciding that because a civil engineer (let's say) is not an obsessive reader of Backhoes Today or Girder Week and Crane Technology--and never invented a new kind of hammer--he must not be working on anything important or cool.
You are asking them to (at least) give away their bread-and-butter, or even to risk prosecution.
Remember though that something stay the same like, good OOD (I use TDD to help drive this), understanding what people want ( :-P ), trying not to be an arrogant arse hat etc.
I think we all go in cycles as required. So here I am reading something by this Hansel guy.
My goal was simply to juxtapose the Loud with the Quiet to perhaps remind my Loud Friends that the world's work is being done by a largely underserved Quiet majority.
They don't need to read blogs, follow people on Twitter because none of these people talk about what they are using.
Surely they also don't have any reason to know about fancy things that we are working on for now such as MVC3, jQuery, NoSQL, cloud etc. since their product is a 20 year old legacy enterprise application. Even on its own forum, you can see only a few new post every week.
So while the 1%ers are blogging about how they just wrote a Phonegap app to talk to a REST service running under Ruby, I have real work to do and a schedule to keep.
It's not that I'm in the dark, I know what's going on around me. I'm in tune with my surroundings. I read blogs, read about the latest/greatest tool, I know what Stack Overflow is, try to stay informed about where and how my industry is moving; however, I just don't feel the need, or desire, to shout from on high about what I know and why I think you should do what I'm doing. It's just not my style. I prefer to be in the background doing my thing without fanfare.
My units motto in the military is "The Quite Professionals." I've carried that over to my civilian career and I like it that way.
The bottom line is that we all just need to get things done. I'm not suggesting that the people who do tweet and blog don't. It's your personality and you do it because you love doing it. And honestly, you're very much needed and appreciated.
I'm grateful for all you folks that do blog. That's how I stay informed while quietly going about my day doing what I love to do.
So thank you to all you visible matter folks out there colliding and creating the energy needed to keep our craft going.
I don't program on ASP in Mexico.
I program on C++ in xXXXXx.
I program embedded devices, which create your stupid cloud.
I personally don't participate in "the cloud" because giving up
your independence is stupid squared.
I don't read blogs, because in the last 10 years there was nothing worth reading.
But when all is said and done, one month I might be helping a client integrate Oracle Forms on a J2EE 1.4 app server with IBM FileNet, the next month doing custom extensions for an existing VB6 app that some customer has, the month after that helping fix a Struts application, and the month after looking at some C# 2.0 code.
In my spare time I play with Scala or Haskell or F# or Clojure or Ruby or C# 4+ or C++11x, but that's just to keep my brain limber. The odds of encountering a client that uses any of this are tiny. More productively I keep abreast of software engineering thought: lean architecture and DCI and modern testing ideas and the latest approaches to requirements gathering. And that I do try out with clients. But I don't really have time to blog about it.
If you develop long enough I think you arrive at a realization that it's solid software engineering principles that matter, and that those don't change rapidly. I think it's cool that a 1% or 5% constantly experiment with tools. I won't deny that JSF or ASP.NET MVC, for example, have made my life easier; but I also remind myself that some years before either of those matured I worked for a company where we churned out solid web apps using ColdFusion. I don't think I've ever run across a programming situation, not once in over 30 years, where you couldn't adequately solve an IT problem with solid non-bleeding-edge tools available at the time.
A corollary to the theme of this article is the relevance of what a lot of the cutting-edge 1% or 5% of developers are doing. Cloud and mobile and NoSQL and all that good stuff is fine and wonderful, but we haven't even come close to solving the basic SDLC loop yet: requirements, design, coding, test, maintenance, in whatever particular methodology you care to employ. The majority of projects still have a hard time completing good solutions on time and within budget. Are all these tools and new infrastructures going to solve that somehow? No. And it's that problem that the "Dark Matter" developers actually deal with - every day.
Please don't tell me that it is not insulting, well-meant and purely theoretical. You are rare case of the speaker guy who actually can have shit done - quick and solid. But for you there are 100 of speaker guys who had no idea how to code another list box, when they have to. And they are all with you shoulder-to-shoulder pointing their finger at those Dark Matter people with derision.
Saying something like that for person who holds a unique position is an ethical equivalent of those who are CEO referring to those who are not as losers. 501 developer works 8 hours to gain his fixed wage.
The nice thing about being a dark programmer is that these things aren't mutually exclusive.
Just sayin ;)
Almost all of the awe-inspiring developers I met who changed my own understanding of programming and professionalism as well had no blogs - barely even an internet presence. Why? I can't be 100% sure, but they always seemed more interested in producing cool stuff than in exchanging ideas.
That being said, evangelism is no doubt important for the lay developer, but I would also divide the 1% pushing things forward in half. One half likes talking about it, the other half doesn't. Both halves are needed to push things forward.
I spoke with Scott Hanselman once and at one point he said "Maybe I will see you at a Code Camp". "What is a Code Camp?", I asked. Scott's response? "Are you a developer?". We had already discussed that I was. Clearly, it was difficult for him to understand how that was possible if I had to ask this question.
It is not exactly as if I was using VB6 in the third world either. For one thing, I had listened to most of Hanselminutes by then. In addition, I had just recently written a .NET compiler for fun, moved a client to distributed version control, completed a sophisticated business app, modified Mono so that I could use <%: %> in ASP.NET 3.5 on Linux, and achieved a rep of over 2000 on Stackoverflow (not much I realize, and now higher, but enough to not be punching out at 5:01). I saw Scott speak at Dev Days in Seattle a couple of weeks later but have still never been to a Code Camp.
My point is not to harsh out on Scott. I admire him greatly and this is another good post. Rather, my point is that even when we know we are in "the 1%" we might be in "the 99%" from another perspective.
Mainframes had black screens reminding of Dark matter.
But there were so little bugs and panic technical situations.
However code was sequential, 100,000+ lines of code at times in a single program. C++ is the basis for modern languages. Newer challenges are created for newer programmers and architects.
What struck me was that he was one of the most cheerful people I had encountered on the train, and he exuded enthusiasm for his work as he explained it to me. He saw his work as vitally important for the success of his business and for his customers. The rest of the trip, he examined his listing with engagement of an avid crossword fan working on the latest puzzle from the New York Times. That encounter left me with a new appreciation for those who are not working on the cutting edge of technology.
I think we have 200 devs in the company I am working for, but never met any of them online.
Still the ones I know are smart motivated engineers that read books, blogs and produce high quality code solving quite complex problems.
Most web applications that we write are in asp.net mvc using latest versions of the ms tech stack.
Our desktop applications are worse off. While WinRT is looking mighty fine, most of our customers will not switch to windows 8 on release. I guess 30% of them are still on XP. So the need to stay current here is much less. It's even painful looking at all the new goodies we cannot use since we still have to PInvoke or use COM. grrrr.
I really like bloggers and speakers and thank you very much for channel9. It's just that our customers have problems were technology is just a small part of the solution.
For me I was interested in new ideas, but I was too busy writing code to get out there. I also did not know about the community and the blogosphere. The only time I peaked outside was if I went to a conference like VSLive or read a technology book like POEAA.
I know it has been six years since I joined Microsoft, but I cannot believe it has changed 'that' much.
I prefer staying in the dark.
Thank you! :)
I enjoyed reading this article but it left me with a bit of a distaste because it says we need to reach out to these other folks and get them to be more like us and join the online community. You sort of caught that and mentioned that we could learn from them how to get stuff done. But we have to be careful when we think our way of using technology or interacting is better. Couldn't it be that we all have different roles in the geek world we live in? And that it is ok for each of us to continue doing what we are doing, whatever that is?
As a few people mentioned, I see that there are different types of folks: Developers, Programmers and Coders.
Devs are like you and me: we try things, we read geek stuff at night, we break stuff and then fix it, we are the ones stepping into new territories and then teaching it at conferences/user groups, blogging about it, etc...and we love it!
Programmers work the 9-5 and then go home...this majority group expects us to continue to blog the things they will use 9-5 the next day. They will take our ideas and tweak them into the practical programming they do. They don't go to conferences or user groups unless they are forced to, the company pays for the trip to Vegas or the topic suits them and their own need/want. They like their role as is.
Coders are another minority, working hard, extra hours as needed, plugging away in the older technologies, happy as a clam, or at least resistant to change.
Each group is happy where they are and it is very difficult getting them to change groups. In fact, it is futile and frustrating to expect them to change. It is detrimental to want them to change...we need idea people, we need technical people to take the ideas and bring them to the practical, and we need the workhorses to bang out code for as long as we need to support the codebase.
Each has their role and we are better served letting each group do what they do, finding the right place for them in our companies (where admittedly some groups won't fit) and putting good leaders over them to help them be the best they can be in their role.
Keep doing what you are doing, since you serve a minority but large group of true developers very well. Thank you! (On a side note I would love to see a post or two on your success in balancing solid technical community involvement with family and personal time. It is inspiring to see such commitment to both sides of life, and for it to work so well. Idea-oriented, forward-looking guys tend to not be so balanced.)
I haven't read the entire comment list but I have some $.02 about the company I currently work with...
We work on a software product which makes use of Datastage TX to handle transforming text from one format into another - we need to process many of these plain text messages in various formats and route them to the appropriate destinatiosn. The people that write the DataStage TX 'code' to transform the text are considered 'Developers'.
Writing 'code' in Datastage is not like writing code in Java/C#/C/C++/Perl/PHP/Python, etc. I guess something somewhat relatable though is 'Developers' at my job look at specifications of the input to expect and make sure they have correct output, again to a certain specification.
Backtrack to when I interviewed for this company I was considered because I have a technical background from school (Comp Sci degree with knowledge of Java) so I would learn the system from a systems analyst point of view then transition to a developer. Little did I know a developer meaent what I described in the paragraph above, not what you guys think of as a software developer who writes code, unit tests it, collaborates with others on problems, code reviews, etc.
I guess my issue is, what qualifies you to be a developer?.
I don't think any of these guys follow the tech industry at all or understand a software stack (from data layer up to the application layer, for instance) like a traditional software developer. These guys may be "Dark Matter Developers", that to be quite frank, would not contribute too much to the online developer community. Maybe I've got a jaded view because I feel misled when I got hired for this job.
Can you see me now? ☺ No? How about now? Still no? Got a 404, right? Wait, how about now?
Now that wasn't very productive, was it?
Dark matter devs would be a good developers life podcast.
The point is this, most off us dont get to use all this new stuff cuz the companies that we work for are not ready to use them, we try to push the envelope for the to use new technologies but like all things in mature companies and organizations, change is slow.
Keep the job perfecting new technologies for us, we a looking and following blogs, and post and all that. We just also have real life with kids, schools, football and all the other stuff that make life worth while, you'll get there one day also and i hope that you have made a dent on moving tech forward.
I found this particular post of yours extremely fascinating. While we're categorizing programmers, I'd like to add my category to the list-- fringe programmers. Maybe we don't have time, resources, or interest to focus on programming as a main priority in our lives and we're ok with that.
Personally, I really enjoy reading your blog because you do give examples from all kinds of different development areas and with a host of different languages. I read articles and blogs about programming as a way to informally school myself at a slow slow slow pace, and I create small application centric scripts once in a while. Learning better logic and looking at more professional code regularly helps me do that to the best of my ability.
I am fascinated that you say you can identify some parts of the seemingly ever changing tech environment that are here to stay. Would you consider doing a blog post on those things that you predict will be around for a while and why? Any of your other readers who are on the fringes (your posts are extremely accessible because you are effectively giving us expertly commented code, in my opinion) like me would probably also appreciate it.
Thank you for the fantastic content.
On the other hand, I love old software: Win XP, VS2008, VB6, notepad for C++... those are my tools :D
Thanks all - from a young developer trying to find his place in the craziness that is the software game.
It's about doing what you get the most out of. Finding what that is is the non-trivial part.
When you maintain an old project based on old technologies you haven't enough time to be online and test all new technologies and features, play with the latest beta etc. Just no time for pushing limits.
It makes me sad that so many shops are infested with these types of developers and just wallow in mediocrity.
http://www.cosmosmagazine.com/news/5765/scientists-shine-light-dark-matter
Some people prefer to see what's going on around them without actually getting involved.
To be honest, it's actually quite difficult to participate online (in my experience) as you're not quite sure which blog/twitter user etc. to follow, let alone comment on, participate in etc.
And I do think the biggest issue here is, as you said, the fact that the bloggers out there are focused on the latest and greatest technology whilst developers are in the middle of their projects using their current stacks, so they wouldn't have anything to comment on.
It's quite intimidating actually, as you could feel so left behind. And I'm sure most devs out there get a bit despondent when a new release comes out especially as they've just managed to get into the previously 'new release'.
It's tricky and actually takes a lot of effort to stay up-to-date, and as the one comment said, a lot of developers just view this as a job.
This post inspired me to get more involved in the developer community. It took me a few months to lessen my contractual workload, but I've finally started a blog (shameless plug: http://www.adamelmore.net/) and I'm on the path towards change. I've always read technology blogs (this being my favorite), but other than that I think I mostly fit the bill.
Thanks for the acknowledgement, Hanselman.
My experience is much the same, I see them "getting things done" and they're really great developers, just not the sharing type like me.
In a non startup-bubble market, in a unpopular language, we manage to have monthly meetings. How is this possible? There's a lot of CF in the 99%. When I try to force a meeting on "modern" techniques (MVC frameworks, new JS frameworks, etc) I get some resistance or complaint. I love new technology but must meet the needs of my community.
tl;dr Better to "old" language that works or cutting edge that isn't even 1.0?
Comments are closed.