The Crowdsourcing of Software Quality
Sponsored By
I posted a rant back in 2012 called "Everything's broken and nobody's upset." Here's another. Lots of people commented on that first post and a number agreed with the general premise. Some were angry and thought that I was picking on particular companies or groups. Sure, it's easy to throw stones, and criticism is a great example of stone throwing. So, in the years since I posted I made a concerted and focused effort on a personal level to report bugs. By this, I mean, I REPORT BUGS. I take screencasts or videos, I email reproductions (repros) and I fill bug issues anywhere and anytime I can because a Bug Report is a Gift.
Fast forward a few years, and I think that we as an industry are perhaps still headed in the wrong way.
Technology companies are outsourcing QA to the customer and we're doing it using frequent updates as an excuse.
This statement isn't specific to Apple, Google, Microsoft or any one organization. It's specific to ALL organizations. The App Store make it easy to update apps. Web Sites are even worse. How often have you been told "clear your cache" which is the 2015 equivalent to "did you turn it on and off again?"
It's too easy to ship crap and it's too easy to update that crap. When I started in software we were lucky to ship every 6 to 9 months. Some places ship every year or two, and others still ship once.
I see folks misusing Scrum and using it as an excuse to be sloppy. They'll add lots of telemetry and use it as an excuse to avoid testing. The excitement and momentum around Unit Testing in the early 2000s has largely taken a back seat to renewed enthusiasm around Continuous Deployment.
But it's not just the fault of technology organizations, is it? It's also our fault - the users. We want it now and we like it beta. We look at software like iOS6 and say "it feels dated." I even overheard someone recently say that iOS9 felt visually dated. It JUST came out. Do we really have to restyle our sites and reship our apps every few months to satisfy a finicky public?
As with many rants, there isn't a good conclusion. I think it's clear this is happening. The question for you, Dear Reader, is do you agree? Do you see it in in your own organization and in the software and hardware that you use every day? Is the advent of the evergreen browser and the always updated phone a good thing or a bad thing?
Sound off in the comments.
Sponsor: Thanks to Infragistics for sponsoring the feed this week. Responsive web design on any browser, any platform and any device with Infragistics jQuery/HTML5 Controls. Get super-charged performance with the world’s fastest HTML5 Grid - Download for free now!
About Scott
Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.
About Newsletter
Hosting By
It's happening, without question. But I see this more as growing pains than any consistent issue. The problem with software is that it grows so fast that it becomes difficult to distinguish those growing pains from actual, systemic problems.
I'm seen people with my title (STE) get converted (SDET and now SE) or get pushed out. Software sucks, and it's getting worse. Perhaps we'll just live with low expectations, like we do of used car salesmen, and fast food.
I see this in my organization with (expensive) commercial off-the-shelf applications frequently. You would think others would have noticed at least some of the bugs and reported them, but I just upgraded a major system to a newer version and had filed three bug reports within two hours. We were one of the last companies to upgrade this product.
I agree with Scott but have to disagree entirely with Matthew. The number of known website hacks of financial institutions, the IRS, the federal govt, stores, etc. alone should make it obvious we have an actual, systemic problem. In the past couple years we have also had firmware bugs in card that resulted in deaths, hackable locks on 4 million hotel room doors, signal boosters to unlock car doors, websites that stream unknowingly exposed baby monitors and home security cameras, hackable home routers and print servers... QA is a serious problem. Until it becomes more profitable to spend time and money on QA than to release it and see what happens, we are in trouble.
Well, I don't like it beta, that's for sure (but I can only speak for myself.) Perhaps the reason is I've grown up in the early 90s when the Internet and cell phones were not as wide-spread, when it was harder to push out updates as frequent as today, and therefore (I assume) companies, developers, organizations etc. put a little more effort in shipping software that was more polished and free of annoying bugs. Looking back at the time twenty years ago, I can't really say that everything turned out better the way it works today.
I don't think it's really the users fault because the user more or less only adapted to the style of a constant update flow. I mean, start pushing out new versions faster and faster, and people will eventually ask for them in shorter and shorter intervals. Another part of the problem is that the mobile market succeeded in establishing the feeling of short-time obsolescence - the users are trained to replace their perfectly working phones every other year, not only because of slightly better hardware that it many cases don't really justify an upgrade, but because the new software won't be available for their old one. Last but not least, it's also a social-psychological problem. This is actually too big of a topic to explain in a few sentences, but the essence is: Mass consumption created the feeling that every desire needs to be instantly satisfied and no demand may be frustrated. (You can read more about this very interesting matter from Erich Fromm.)
Now, the problem with bug reports is: Although they should be a gift, I feel that many software providers just don't care to receive them. I just think of all the people who posted about bugs in Visual Studio on Connect and were angry because they felt they were ignored, remained unheard, and the issues were never really solved most of the time. Besides that, I don't think it's the users responsibility to deal with bugs more than necessary. It's OK to have some kind of automatic error reporting, a form where the user just has to click a button to submit it, but why should I do more than that? Am I a beta tester? Am I paid for it? No, and I don't like to play that role. Especially when I pay for software where I expect it to work flawlessly. I'm even ready to wait a little longer until the problems got fixed instead of dealing with an unfinished product, but I guess most people aren't (anymore).
So, TL;DR: No, you don't have to restyle your website or reship your apps to satisfy me. Perhaps the fault of the companies/developers was that they have put the bar too high on their own, set themselves on a cadence that is just too fast to deliver proper results, but now people expect them to deliver in time anyways.
I don't think it's really the users fault because the user more or less only adapted to the style of a constant update flow. I mean, start pushing out new versions faster and faster, and people will eventually ask for them in shorter and shorter intervals. Another part of the problem is that the mobile market succeeded in establishing the feeling of short-time obsolescence - the users are trained to replace their perfectly working phones every other year, not only because of slightly better hardware that it many cases don't really justify an upgrade, but because the new software won't be available for their old one. Last but not least, it's also a social-psychological problem. This is actually too big of a topic to explain in a few sentences, but the essence is: Mass consumption created the feeling that every desire needs to be instantly satisfied and no demand may be frustrated. (You can read more about this very interesting matter from Erich Fromm.)
Now, the problem with bug reports is: Although they should be a gift, I feel that many software providers just don't care to receive them. I just think of all the people who posted about bugs in Visual Studio on Connect and were angry because they felt they were ignored, remained unheard, and the issues were never really solved most of the time. Besides that, I don't think it's the users responsibility to deal with bugs more than necessary. It's OK to have some kind of automatic error reporting, a form where the user just has to click a button to submit it, but why should I do more than that? Am I a beta tester? Am I paid for it? No, and I don't like to play that role. Especially when I pay for software where I expect it to work flawlessly. I'm even ready to wait a little longer until the problems got fixed instead of dealing with an unfinished product, but I guess most people aren't (anymore).
So, TL;DR: No, you don't have to restyle your website or reship your apps to satisfy me. Perhaps the fault of the companies/developers was that they have put the bar too high on their own, set themselves on a cadence that is just too fast to deliver proper results, but now people expect them to deliver in time anyways.
I don't think many small bugs are a problem in the case of most web apps. There are obviously exceptions (banking, security etc.). Software these days is going to be sloppy because the industry is extremely competitive, users are hungry for features, and software is complex and has to run on multiple platforms. I think web companies are investing less in QA and testing because users really don't care. You might have found some edge case you find annoying but maybe it only affects 0.01% of users. There's just no value in fixing something like this. The user refreshes, finds a workaround and hopes it gets fixed in the next release.
Not sure if I do agree entirely, but I don't disagree :)
I tend to have the most problems with software that has slower release cycles. The odd thing is, in many cases you can adapt to whatever the "bug" happens to be. Often it's nicer to get a continuous stream of improvements with a few hassles occasionally, than static software that doesn't change. Of course not all software can do this, and I think in the world of software, there is no one "moral" way of developing software. We are post modernists where the measures of "good" are deconstructed and reinvented regularly.
But we can definitely improve! Software is an evolutionary process...
I was in the "In" group of extreme programmers since 1999, and followed the whole evolution of agile, and seeing it become mainstream. I've seen how people have gone crazy with unit testing, continuous delivery, etc.... Part of the problem is a lot of the "agile" people were consultants, and effectively "sold" a package that people should use. I kind of distant myself from agile now as it kind of doesn't represent the message I got from the main instigators, and the message I got is what will help improve the situation.
I think the important thing was:-
Think Clearly.
Which may seem obvious, or easy to say “well of course I do that”, but seeing the likes of Kent Beck doing this, you start getting an idea that maybe you don’t. It’s almost a philosophical approach to things. It also tends to produce a lot of counter intuitive advice. It’s kind of hard to sum up exactly what this ‘thinking clearly’ looks like, but it’s sort of along these lines :-
Think what is the essence of what you are trying to do, strip away all the means to an end, keep your actual ends clear in your mind.
The various practices of agile, and software in general are mostly a means to an end. They get invented AND discarded depending on ones ends.
Some of the dogma of agile was to really try and get people to engage (there was a lot of resistance) with counter intuitive practices and use them so they can develop an understanding of some of the means to an end. A lot of the dogma has stayed and driven people to do things that are really a bit weird.
What we really need to do is focus on our ends. So what is an end? It depends...open source software is likely to be quite different than commercial software. But a common trait of an end is :-
Working software at the right time
TDD is a means to an end to help with getting working, and help getting the right time, and also help with structuring how to do it.
Continuous Delivery is a means to an end to help with feedback to to ensure we are doing the right thing, and we sustainably getting somewhere.
Functional Programming is a means that allows simpler expression of functionality that is easy to verify
These things are just means to an end. There’s lots of means, and some really smart people contributing all kinds of great means. The philosophically reflective start noticing things common about some of the more widely useful things, like “Simplicity”, “Collaboration”, “Feedback”, “Clarity of Intent”, “Learning” etc.
But… they are just a means to an end, and anyone is free to find new means to an end through Clearly Thinking about what they are doing. Which coming to the crux of the matter, what I’m trying to say :) There is no formula for software, there’s no magic, there’s a lot of existing collective learning, and there’s lots of great means, and many more to come, but to get better we need to be clearly thinking about the means we are using and why, we need to be creative about our means, and actively evolve what we are doing through clear thought.
I tend to have the most problems with software that has slower release cycles. The odd thing is, in many cases you can adapt to whatever the "bug" happens to be. Often it's nicer to get a continuous stream of improvements with a few hassles occasionally, than static software that doesn't change. Of course not all software can do this, and I think in the world of software, there is no one "moral" way of developing software. We are post modernists where the measures of "good" are deconstructed and reinvented regularly.
But we can definitely improve! Software is an evolutionary process...
I was in the "In" group of extreme programmers since 1999, and followed the whole evolution of agile, and seeing it become mainstream. I've seen how people have gone crazy with unit testing, continuous delivery, etc.... Part of the problem is a lot of the "agile" people were consultants, and effectively "sold" a package that people should use. I kind of distant myself from agile now as it kind of doesn't represent the message I got from the main instigators, and the message I got is what will help improve the situation.
I think the important thing was:-
Think Clearly.
Which may seem obvious, or easy to say “well of course I do that”, but seeing the likes of Kent Beck doing this, you start getting an idea that maybe you don’t. It’s almost a philosophical approach to things. It also tends to produce a lot of counter intuitive advice. It’s kind of hard to sum up exactly what this ‘thinking clearly’ looks like, but it’s sort of along these lines :-
Think what is the essence of what you are trying to do, strip away all the means to an end, keep your actual ends clear in your mind.
The various practices of agile, and software in general are mostly a means to an end. They get invented AND discarded depending on ones ends.
Some of the dogma of agile was to really try and get people to engage (there was a lot of resistance) with counter intuitive practices and use them so they can develop an understanding of some of the means to an end. A lot of the dogma has stayed and driven people to do things that are really a bit weird.
What we really need to do is focus on our ends. So what is an end? It depends...open source software is likely to be quite different than commercial software. But a common trait of an end is :-
Working software at the right time
TDD is a means to an end to help with getting working, and help getting the right time, and also help with structuring how to do it.
Continuous Delivery is a means to an end to help with feedback to to ensure we are doing the right thing, and we sustainably getting somewhere.
Functional Programming is a means that allows simpler expression of functionality that is easy to verify
These things are just means to an end. There’s lots of means, and some really smart people contributing all kinds of great means. The philosophically reflective start noticing things common about some of the more widely useful things, like “Simplicity”, “Collaboration”, “Feedback”, “Clarity of Intent”, “Learning” etc.
But… they are just a means to an end, and anyone is free to find new means to an end through Clearly Thinking about what they are doing. Which coming to the crux of the matter, what I’m trying to say :) There is no formula for software, there’s no magic, there’s a lot of existing collective learning, and there’s lots of great means, and many more to come, but to get better we need to be clearly thinking about the means we are using and why, we need to be creative about our means, and actively evolve what we are doing through clear thought.
I think it comes down to lack of discipline. If you're not disciplined in your testing bugs will leak through. Releasing more frequently just increases the need for that discipline.
That said, I'm not saying it's easy. Discipline can be really difficult in a large, complex system and org. As can achieving effective test coverage for what you're shipping.
Iterating fast and releasing often is good, but decent automated test coverage should really be a precursor to a team doing continuous delivery or you're just asking for bugs to slip through.
That said, I'm not saying it's easy. Discipline can be really difficult in a large, complex system and org. As can achieving effective test coverage for what you're shipping.
Iterating fast and releasing often is good, but decent automated test coverage should really be a precursor to a team doing continuous delivery or you're just asking for bugs to slip through.
Couldn't agree more. I work as a software developer. A few weeks ago my selenium tests stopped working in Chrome. Something had changed when Chrome was silently updated. That time upgrading the selenium webdriver solved the problem. Last week clicking some links in IE brooke, turned out that a minor update of a major SAP framework introduced a bug when building the frontend code in (node) development mode. So the solution is not to always lock the version of your dependencies, and not to always stay up to date. And it is almost a full time job to follow releases and commits close enough for 30+ dependencies.
It's a struggle to keep all those problems away from my users...and we havn't even gotten to the code I actually write. The environment is more complex today then 5 years ago and the number of external dependencies seems to increase every day.
It's a struggle to keep all those problems away from my users...and we havn't even gotten to the code I actually write. The environment is more complex today then 5 years ago and the number of external dependencies seems to increase every day.
"did you turn it on and off again?"
Don't you mean the other way around? Did you turn it off and on again?
Don't you mean the other way around? Did you turn it off and on again?
I don't think it's true that it's a new problem or a problem caused by the fact it's easier to update software. Think of some of the great rendering bugs IE6 had for example.
There's always going to be a bell curve of software quality and even the best software will always have the occasional bug. Software gets complicated quickly and people have a limit on the amount of complexity they can handle. The best we can do is look for ways to get better at testing more of our software and managing the complexity.
I think in the past 15 years since I've been developing software we have gotten better at both. It's staggering the components we have to build things out of today which are often available for free. The technology that we have for testing is also much better and I think the average team has also come a long way in how their testing is done.
There's always going to be a bell curve of software quality and even the best software will always have the occasional bug. Software gets complicated quickly and people have a limit on the amount of complexity they can handle. The best we can do is look for ways to get better at testing more of our software and managing the complexity.
I think in the past 15 years since I've been developing software we have gotten better at both. It's staggering the components we have to build things out of today which are often available for free. The technology that we have for testing is also much better and I think the average team has also come a long way in how their testing is done.
The thing is that consumers are used to fast updates and expect developers to deliver fast. They have no clue how software development works. Same for managers in companies. They only look at deadlines and delivery and the github ship it squirrel. http://shipitsquirrel.github.io/
Having high pressure deadlines impacts the quality of software. Speaking for myself, I get feature requests and people want them live on the same day. When I put up test builds, people do not have the time to look at it. I only get feedback from live/production builds. Which is often fixed in the new test builds. But no one looks at test stuff.
I do run windows insider builds on both desktop and phone and provide feedback through the feedback app. I totally agree that prosumers and developers should file more bug reports or send more feedback even when it is not their software. I am also using snip from microsoft office garage. Which is great, but beta. You see new stuff rolling in often. I think that when developers open up their beta software, it is a gift that you have early access and you are obligated to return a favor with bug reports.
Having high pressure deadlines impacts the quality of software. Speaking for myself, I get feature requests and people want them live on the same day. When I put up test builds, people do not have the time to look at it. I only get feedback from live/production builds. Which is often fixed in the new test builds. But no one looks at test stuff.
I do run windows insider builds on both desktop and phone and provide feedback through the feedback app. I totally agree that prosumers and developers should file more bug reports or send more feedback even when it is not their software. I am also using snip from microsoft office garage. Which is great, but beta. You see new stuff rolling in often. I think that when developers open up their beta software, it is a gift that you have early access and you are obligated to return a favor with bug reports.
I agree - it's not only bugs, but the way software will lose significant features at the whims of product teams to facilitate some arbitrary redesign or rewrite. There's one music service in particular where I'm not entirely sure if I'm a customer or an unpaid regression tester.
I think frequent release cycles are an excuse often used, but one of the significant reasons behind it is that the cost of throwing out slapdash software has become less obvious. Back in the 1990s, if you put a buggy piece of software on your gold master it had a very real and noticeable cost in mailing out patch disks to the affected customers. If your software lost a feature between versions, people didn't buy the update. Correcting the impact of a poor-quality release had a bill of materials, and there were enough high-profile cases of buggy releases going very badly wrong (anyone remember the Frontier: First Encounters debacle?) that software companies had a heavy incentive to make sure what was in the box actually worked.
The cost now isn't as easy to identify. It's still there: there's a certain brand of consumer electronics I simply won't buy any more due to their consistent shipping of sloppily-QA'd updates... but you're only guessing at what the impact on your sales or reputational damage is, not least because people are getting their support via forums (or giving up) rather than making calls to your customer service. You don't get the obvious, "we got 6,000 calls and had to send 3,500 patch disks at a cost of $3 each to fix this problem" feedback any more. Fluid versioning and automatic updates mean you aren't able to say, "nobody's buying v4 after we dropped feature x".
That kind of supports your point, in a roundabout way. Us users are partly to blame; not only do we demand these constant updates but we don't tell companies when they don't work, we search for answers, maybe moan about it on Twitter then buy someone else's product on our next upgrade cycle. We need to get better at sharing our frustrations, and telling companies when their software quality falls below our expectations. It's one of the reasons I like Microsoft's ":)" icon in Visual Studio and the way Windows 10 asks you how your update went - it's a reminder that you can tell the company if you're annoyed by a newly-introduce bug, and get back to that world where the impact of releasing low-quality software is obvious.
I think frequent release cycles are an excuse often used, but one of the significant reasons behind it is that the cost of throwing out slapdash software has become less obvious. Back in the 1990s, if you put a buggy piece of software on your gold master it had a very real and noticeable cost in mailing out patch disks to the affected customers. If your software lost a feature between versions, people didn't buy the update. Correcting the impact of a poor-quality release had a bill of materials, and there were enough high-profile cases of buggy releases going very badly wrong (anyone remember the Frontier: First Encounters debacle?) that software companies had a heavy incentive to make sure what was in the box actually worked.
The cost now isn't as easy to identify. It's still there: there's a certain brand of consumer electronics I simply won't buy any more due to their consistent shipping of sloppily-QA'd updates... but you're only guessing at what the impact on your sales or reputational damage is, not least because people are getting their support via forums (or giving up) rather than making calls to your customer service. You don't get the obvious, "we got 6,000 calls and had to send 3,500 patch disks at a cost of $3 each to fix this problem" feedback any more. Fluid versioning and automatic updates mean you aren't able to say, "nobody's buying v4 after we dropped feature x".
That kind of supports your point, in a roundabout way. Us users are partly to blame; not only do we demand these constant updates but we don't tell companies when they don't work, we search for answers, maybe moan about it on Twitter then buy someone else's product on our next upgrade cycle. We need to get better at sharing our frustrations, and telling companies when their software quality falls below our expectations. It's one of the reasons I like Microsoft's ":)" icon in Visual Studio and the way Windows 10 asks you how your update went - it's a reminder that you can tell the company if you're annoyed by a newly-introduce bug, and get back to that world where the impact of releasing low-quality software is obvious.
Yes, this is happening, but as JP Hellemons says, that's because it's what the customers expect, and if you don't deliver, well, they'll take their money somewhere else. I think it's very sad - they're paying good money for my experience (not just my code) and then not trusting me when I say 'hold on a minute, we need to check the quality on this'.
It's also not helped by the whole agile manifesto being heard by some as "We can go faster! Don't think up front, just do it! Deliver!". I know, that isn't what agile is supposed to be about, but that's how it's heard by the people with the purse strings. Agile is also just about the worst name for it, too - it sounds rushing, speedy.
I think that another part of the problem is that most software is not proper engineering (There are exceptions). Mostly, it's stuff that's hacked together, and quality control comes down to the quality of the coder, what their day's been like, whether they've been well briefed, etc.. You wouldn't dream of, say, building a bridge (even a little one) based on a drawing on the back of an envelope - but that's pretty much what a lot of websites are like. You certainly wouldn't expect to have that bridge finished by the end of tomorrow - but that's normal in website development.
Until we stop hacking code together and start treating software as proper engineering, quality will suffer. But that'll make things slower and more expensive - so it'll never happen for most purposes.
I guess it's the old saying - "You can have fast, cheap or good. Pick two".
It's also not helped by the whole agile manifesto being heard by some as "We can go faster! Don't think up front, just do it! Deliver!". I know, that isn't what agile is supposed to be about, but that's how it's heard by the people with the purse strings. Agile is also just about the worst name for it, too - it sounds rushing, speedy.
I think that another part of the problem is that most software is not proper engineering (There are exceptions). Mostly, it's stuff that's hacked together, and quality control comes down to the quality of the coder, what their day's been like, whether they've been well briefed, etc.. You wouldn't dream of, say, building a bridge (even a little one) based on a drawing on the back of an envelope - but that's pretty much what a lot of websites are like. You certainly wouldn't expect to have that bridge finished by the end of tomorrow - but that's normal in website development.
Until we stop hacking code together and start treating software as proper engineering, quality will suffer. But that'll make things slower and more expensive - so it'll never happen for most purposes.
I guess it's the old saying - "You can have fast, cheap or good. Pick two".
Well,
First I'm 100% at your side! Maybe should test a LOT more and create better ways to catch that error and send back to home, not to launch more versions but to INCREASE our base of tests.
My old boss used to hate me when I was trying to hit 100% code cover, but even with 95% covered we still receive errors which as not catch in one simple unit test, more complex scenarios always escapes us when programming, this is why I think a specialist in test should be always part of the time.
First I'm 100% at your side! Maybe should test a LOT more and create better ways to catch that error and send back to home, not to launch more versions but to INCREASE our base of tests.
My old boss used to hate me when I was trying to hit 100% code cover, but even with 95% covered we still receive errors which as not catch in one simple unit test, more complex scenarios always escapes us when programming, this is why I think a specialist in test should be always part of the time.
What I hate the most is when a working website ships an update which breaks something, and you don't even get a choice of reverting to the working version. This happened on GitHub a few months ago, and it cost me productivity for a few days - and GitHub still gets some fundamental things wrong like detecting which commits have already been merged, if the branch history is too long.
Even programs like Resharper, which I use every day, has had some bugs that have existed over multiple versions (and they just announced their latest v10 critical bugs: http://blog.jetbrains.com/dotnet/2015/11/09/resharper-10-critical-issues-and-fix-schedule/).
"Google Now" constantly vibrates my phone (about once a week) to remind me that "current traffic is heavy - your journey home will take an extra 12 minutes!" during rush hour, despite the fact I never drive during rush hour and it should be perfectly capable of knowing when I leave work.
I think you are right that there is a lack of a centralised bug reporting system. As a developer, I feel like I can spot a "real" bug but often I have no way of shouting "this is a real bug, I know what i'm talking about!". Reporting a bug is often a really arduous process. I have been involved a lot in bug reporting for TortoiseGit, and I can either report a kind-of-vague bug (which I know is definitely a bug), in about 3 minutes, or I can set up an open-source GitHub repository and nail the exact reproduction instructions, which will probably take me an hour. I'm not going to spend an hour of my time reporting some stupid uppercase/lowercase issue that some screen doesnt deal with, why can't they just BELIEVE ME in the first place! (I understand this swings both ways - as a developer I dislike vague bug reports with no repro instructions.)
These days when I actually have some working software, i will fight tooth and nail against people asking me to update it, because it's so rare and the chances of an update breaking something are so high.
Even programs like Resharper, which I use every day, has had some bugs that have existed over multiple versions (and they just announced their latest v10 critical bugs: http://blog.jetbrains.com/dotnet/2015/11/09/resharper-10-critical-issues-and-fix-schedule/).
"Google Now" constantly vibrates my phone (about once a week) to remind me that "current traffic is heavy - your journey home will take an extra 12 minutes!" during rush hour, despite the fact I never drive during rush hour and it should be perfectly capable of knowing when I leave work.
I think you are right that there is a lack of a centralised bug reporting system. As a developer, I feel like I can spot a "real" bug but often I have no way of shouting "this is a real bug, I know what i'm talking about!". Reporting a bug is often a really arduous process. I have been involved a lot in bug reporting for TortoiseGit, and I can either report a kind-of-vague bug (which I know is definitely a bug), in about 3 minutes, or I can set up an open-source GitHub repository and nail the exact reproduction instructions, which will probably take me an hour. I'm not going to spend an hour of my time reporting some stupid uppercase/lowercase issue that some screen doesnt deal with, why can't they just BELIEVE ME in the first place! (I understand this swings both ways - as a developer I dislike vague bug reports with no repro instructions.)
These days when I actually have some working software, i will fight tooth and nail against people asking me to update it, because it's so rare and the chances of an update breaking something are so high.
Continuous Deployment can be used for good and for bad. It means you refine and improve the product in quick manner.
It can also be used as excuse to release crap, then fix it later.
Continuous deployment operations should have a lot automated testing, unit tests, coded UI tests to keep the quality up.
It can also be used as excuse to release crap, then fix it later.
Continuous deployment operations should have a lot automated testing, unit tests, coded UI tests to keep the quality up.
I think it's great!
It just means that it's that it's easier for people who give half a crap to shine all the brighter. I mean, with the bar that low its not going to take much is it?
It is a problem and it is getting worse. Why, just today I had a "disagreement" with a colleague who is yet to see the benefits of all the testing we undertake. Unit tests, integration tests and then acceptance tests; why the hell do we bother? "I don't believe it's helping us be any better". Never mind the fact that we're the ONLY team who continue to ship (quality) on time. I mean, had I taken my longsword to work, he would have been headless.
And the way we have swung from "TEST ALL THE THINGS!!" to ship it and don't worry about jack diddly... We'll just execute our massively automated crap producing deployment constantly and "she'll be 'right mate!" as they say here down under. It's enough to make me cry.
Well, I don't buy it. That lot can go play in busy traffic. Its unprofessional, embarrassing and massively uninspiring. It pretty much lines up with everything these days though if you think about it... "instant everything".
I for one will remain a craftsman to the end. Keep pushing the bar down n00bs, I'm looking better for it every day!
It just means that it's that it's easier for people who give half a crap to shine all the brighter. I mean, with the bar that low its not going to take much is it?
It is a problem and it is getting worse. Why, just today I had a "disagreement" with a colleague who is yet to see the benefits of all the testing we undertake. Unit tests, integration tests and then acceptance tests; why the hell do we bother? "I don't believe it's helping us be any better". Never mind the fact that we're the ONLY team who continue to ship (quality) on time. I mean, had I taken my longsword to work, he would have been headless.
And the way we have swung from "TEST ALL THE THINGS!!" to ship it and don't worry about jack diddly... We'll just execute our massively automated crap producing deployment constantly and "she'll be 'right mate!" as they say here down under. It's enough to make me cry.
Well, I don't buy it. That lot can go play in busy traffic. Its unprofessional, embarrassing and massively uninspiring. It pretty much lines up with everything these days though if you think about it... "instant everything".
I for one will remain a craftsman to the end. Keep pushing the bar down n00bs, I'm looking better for it every day!
The issue, imho, is that we are still lacking credibility on functional test suites. This makes people tend to rather test directly on production environment. Hopefully,
this will improve with the rise of API middleware mocking services.
Thank you Scott for being a Good Samaritan submitting bug reports, I will follow your example!
this will improve with the rise of API middleware mocking services.
Thank you Scott for being a Good Samaritan submitting bug reports, I will follow your example!
I think you're right, it is a problem. I wonder if this is a generation thing too, people used to fear or at least dislike change. For me I like consistency, I don't want a constant re-design, I want to learn a pattern and keep using it (unless it's broken), not changing for changes sake. Which ultimately is part of a good user experience.
But I do think we're getting used to poor quality. I think the lack of company responses to bugs doesn't help. When you've got threads complaining about a bug but no official response or commitment to fixing, consumers just have to get used to sub standard quality. This seems especially true for hardware - where we seem to expect a short lifetime of goods/appliances and have to replace them. If there's an issue in some hardware manufacturers often making it very hard/expensive to get it fixed, so again unfortunately we just learn to live with it.
But I do think we're getting used to poor quality. I think the lack of company responses to bugs doesn't help. When you've got threads complaining about a bug but no official response or commitment to fixing, consumers just have to get used to sub standard quality. This seems especially true for hardware - where we seem to expect a short lifetime of goods/appliances and have to replace them. If there's an issue in some hardware manufacturers often making it very hard/expensive to get it fixed, so again unfortunately we just learn to live with it.
I agree -- it's definitely happening, and I also agree to a certain extent it is driven by the fast food mentality of the need to constantly ship new features yesterday.
But when I also think about it, as aseasoned, crusty senior developer with over 25 years in the industry, I am often APPALLED at the lack of skill and attention to detail I see in QA people. I mean, I have enough to do already focusing on my work and the work of my team, that I shouldn't also have to focus on reviewing QA test plans and cases for veracity because bugs slip through to production.
I'm not even talking about the multitude of companies out there that seem willing to forego a formal QA function altogether -- I'm talking about the rare few that actually see the need and dedicate resources to it.
What makes for a good QA person? What makes someone go into that line of work? Why aren't there any (that I know of) college courses/degrees, certifications, or training on how to perform QA? These are the mysteries of life.
But when I also think about it, as a
I'm not even talking about the multitude of companies out there that seem willing to forego a formal QA function altogether -- I'm talking about the rare few that actually see the need and dedicate resources to it.
What makes for a good QA person? What makes someone go into that line of work? Why aren't there any (that I know of) college courses/degrees, certifications, or training on how to perform QA? These are the mysteries of life.
Wow, ton of long great comments. I only have two short points to make:
1) When companies rigidly stick to a SCRUM format with sprint points and burndown charts, it makes development sloppy. Instead of building quality and unit tests, devs are pressured to deliver features that were arbitrarily sized without taking planning and testing into consideration. If the burndown isn't met, Agile is failing. So, devs are left working overtime because of complexity that wasn't originally considered -- and overtime correlates to not writing unit tests. At least that's my experience.
2) The web needs to slow down. Things are changing so rapidly in evergreen browsers, which is nice, but enterprises are slow and I'm sure they will cling to IE until it's pried through their hands. Not to mention there are many countries with emerging markets that are NOT allowed to run Chrome (hi China!) or Firefox. So if you plan to market to China, make sure you're coding semigreen from a few years back.
I'm working on a personal site to promote myself as a brand (kinda like this site!). My burning question is, should I use flexbox? It will make my life easier, and it's super rad and new and if you ask the "CSS professionals" there is "no reason not to use flexbox now!" Except IE10 supports a different syntax. IE9 doesn't support it. And guess what, if I'm creating a site to to show to potential employers, they're probably stuck with IE9 behind their company firewall.
I love that people are trying to innovate with the web, but there needs to be certain points to when we can start integrating this stuff. I'd rather wait until the W3C comes out with a full spec than use polyfills that will be outdated in a year or so. That's more work for me.
1) When companies rigidly stick to a SCRUM format with sprint points and burndown charts, it makes development sloppy. Instead of building quality and unit tests, devs are pressured to deliver features that were arbitrarily sized without taking planning and testing into consideration. If the burndown isn't met, Agile is failing. So, devs are left working overtime because of complexity that wasn't originally considered -- and overtime correlates to not writing unit tests. At least that's my experience.
2) The web needs to slow down. Things are changing so rapidly in evergreen browsers, which is nice, but enterprises are slow and I'm sure they will cling to IE until it's pried through their hands. Not to mention there are many countries with emerging markets that are NOT allowed to run Chrome (hi China!) or Firefox. So if you plan to market to China, make sure you're coding semigreen from a few years back.
I'm working on a personal site to promote myself as a brand (kinda like this site!). My burning question is, should I use flexbox? It will make my life easier, and it's super rad and new and if you ask the "CSS professionals" there is "no reason not to use flexbox now!" Except IE10 supports a different syntax. IE9 doesn't support it. And guess what, if I'm creating a site to to show to potential employers, they're probably stuck with IE9 behind their company firewall.
I love that people are trying to innovate with the web, but there needs to be certain points to when we can start integrating this stuff. I'd rather wait until the W3C comes out with a full spec than use polyfills that will be outdated in a year or so. That's more work for me.
As long as the app is usable and customers are happy no need to dig after every tiny bug . I see it as maturity in thinking in software development world .
Ironic that you make this post a few days after Fallout 4 comes out- Bethsoft is infamous for shipping buggy games and leaving it to the community to fix the bugs themselves- the ultimate crowdsourcing maneuver.
I think you place the blame in the wrong place, Scott. Most often its not software developers making the decision to ship buggy software; its the suits and bean counters who advise them making the decisions. And they're more worried about the share price and exceeding Q4 projections than making a solid release.
I think you place the blame in the wrong place, Scott. Most often its not software developers making the decision to ship buggy software; its the suits and bean counters who advise them making the decisions. And they're more worried about the share price and exceeding Q4 projections than making a solid release.
Several people bringing up the old "The web is changing too fast" junk.
No.
Browsers that are garbage exist. If people want to use garbage, they should expect garbage in return.
If YOU want to support an old, outdated browser, that's fine. Don't try to hold up the security and stability the rest of us want because you made the conscious choice to support out-of-date software.
One huge reason for security flaws in new software is the push to never stop supporting old software; if you choose to support an old version of IE, maybe you don't update to the newest version of TLS, or maybe you continue using an outdated encryption method, or maybe you avoid using modern XSS prevention.
Anecdotally: Getting my company to stop supporting WinXP and IE7 was a nightmare, right up until a few security flaws were discovered which looked like they might harm the business -- then we spent ten times as much updating in the span of three months than we would have if we'd been updating slowly over the past decade.
No.
Browsers that are garbage exist. If people want to use garbage, they should expect garbage in return.
If YOU want to support an old, outdated browser, that's fine. Don't try to hold up the security and stability the rest of us want because you made the conscious choice to support out-of-date software.
One huge reason for security flaws in new software is the push to never stop supporting old software; if you choose to support an old version of IE, maybe you don't update to the newest version of TLS, or maybe you continue using an outdated encryption method, or maybe you avoid using modern XSS prevention.
Anecdotally: Getting my company to stop supporting WinXP and IE7 was a nightmare, right up until a few security flaws were discovered which looked like they might harm the business -- then we spent ten times as much updating in the span of three months than we would have if we'd been updating slowly over the past decade.
Wasaga Beach, Ontario 2015-11-12
this is not new; Microsoft shipped many bugs.
a rose by any other name is still a rose.
types of what i call bugs (you may prefer to call them roses) include code bugs, design flaws, documentation flaws, et cetera
many companies apparently do not care.
examples
MSDN documentation has had code bugs that others and myself have reported in the formerly available community content; the next versions of MSDN left those comments behind but failed to fix the code bugs.
Now one no longer can add community comment to MSDN ... i've reported this to ScottGu and ScottHa; ScottHa has forwarded my reports ... to date, nothing has happened. Compare these two links:
https://msdn.microsoft.com/en-us/library/zt39148a(v=vs.110).aspx
https://msdn.microsoft.com/en-us/library/zt39148a(v=vs.100).aspx
PayPal's REST API documentation imho sucked ... i advised PayPal about the FREE documentation that i created after struggling through PayPal's flawed documentation ... PayPal could have reviewed my documentation and told its developer community but did not; when i complained via Twitter, PayPal simply blocked me from Twitter.
QA people i've worked with are often insufficiently diligent ... i'd get a bug to fix and immediately spot several others that the QA person could have seen in the few minutes that it took me to notice.
MY BIGGEST PET PEEVE
Most .NET code is written without using TDD;
most .NET code is written without using after the fact unit tests;
many who do write unit tests are using MSTest instead of xUnit.net;
many who do write unit tests do not write enough unit tests ...
if the ratio is only 1::1 that's likely too few since most code
has a purpose and at least one fringe condition, ergo 2 unit tests
to 1 chunk of real code should be a starting point.
Worse, many unit tests are silly and written to satisfy a metric rather that thoughtful and written to bullet-proof code; the result creates a nightmare of maintaining the unit tests rather that leaving quality unit tests alone and crafting quality production code instead.
MY SECOND BIGGEST PET PEEVE
why isn't your grandmother doing usability testing on your web and desktop applications?
imho Steve Krugs' "Don't Make Me Think" should be required reading for 100% of programmers, developers, and managers. http://sensible.com/
this is not new; Microsoft shipped many bugs.
a rose by any other name is still a rose.
types of what i call bugs (you may prefer to call them roses) include code bugs, design flaws, documentation flaws, et cetera
many companies apparently do not care.
examples
MSDN documentation has had code bugs that others and myself have reported in the formerly available community content; the next versions of MSDN left those comments behind but failed to fix the code bugs.
Now one no longer can add community comment to MSDN ... i've reported this to ScottGu and ScottHa; ScottHa has forwarded my reports ... to date, nothing has happened. Compare these two links:
https://msdn.microsoft.com/en-us/library/zt39148a(v=vs.110).aspx
https://msdn.microsoft.com/en-us/library/zt39148a(v=vs.100).aspx
PayPal's REST API documentation imho sucked ... i advised PayPal about the FREE documentation that i created after struggling through PayPal's flawed documentation ... PayPal could have reviewed my documentation and told its developer community but did not; when i complained via Twitter, PayPal simply blocked me from Twitter.
QA people i've worked with are often insufficiently diligent ... i'd get a bug to fix and immediately spot several others that the QA person could have seen in the few minutes that it took me to notice.
MY BIGGEST PET PEEVE
Most .NET code is written without using TDD;
most .NET code is written without using after the fact unit tests;
many who do write unit tests are using MSTest instead of xUnit.net;
many who do write unit tests do not write enough unit tests ...
if the ratio is only 1::1 that's likely too few since most code
has a purpose and at least one fringe condition, ergo 2 unit tests
to 1 chunk of real code should be a starting point.
Worse, many unit tests are silly and written to satisfy a metric rather that thoughtful and written to bullet-proof code; the result creates a nightmare of maintaining the unit tests rather that leaving quality unit tests alone and crafting quality production code instead.
MY SECOND BIGGEST PET PEEVE
why isn't your grandmother doing usability testing on your web and desktop applications?
imho Steve Krugs' "Don't Make Me Think" should be required reading for 100% of programmers, developers, and managers. http://sensible.com/
@Scott Slack because most organizations don't have the slighest idea about how to do Quality ASSURANCE (prevent things from ever breaking in the first place, when something does break analyze what went wrong in the software PROCESS that allowed it and prevent it repeating).
Most companies rarely even do Quality Control and some will even actively ship knowningly defective software in the sake of "going fast".
Most companies rarely even do Quality Control and some will even actively ship knowningly defective software in the sake of "going fast".
I think the different viewpoints here are interesting. It shows that experiences do vary. What I see as that software teams that are young have a hard time seeing the pitfalls of what is the latest best-practice or buzz methodology. I see that some don't understand that evergreen and continuous deployment mean that you need to tighten that sphincter and be more methodical.
If you do you will architect more upfront and stick to it for consistency and supportability. Then at some point you will do a system wide refactor tantamount to a rewrite like Entity Framework 7. I just recently uninstalled Chrome because they (the developers of Chrome) didn't respect my need for stability. Not because it is evergreen. You can still have CD or be evergreen without interrupting the user, but you have to care and be disciplined.
Security and stability MUST be a reflex. This be comes more imperative depending on your audience. If you have just built your first app, a little mobile app or enterprise tool, or you manage a large gov website and don't push your self to get better I see a nice job at Game Stop in your future.
If you do you will architect more upfront and stick to it for consistency and supportability. Then at some point you will do a system wide refactor tantamount to a rewrite like Entity Framework 7. I just recently uninstalled Chrome because they (the developers of Chrome) didn't respect my need for stability. Not because it is evergreen. You can still have CD or be evergreen without interrupting the user, but you have to care and be disciplined.
Security and stability MUST be a reflex. This be comes more imperative depending on your audience. If you have just built your first app, a little mobile app or enterprise tool, or you manage a large gov website and don't push your self to get better I see a nice job at Game Stop in your future.
Next article: "The Crowd-Sourcing of Blog Post Quality" -- needs more cowbell in the comments section!
Just kidding, kind-of. At least people are jumping in with genuinely useful discussion. Time to delivery of new features winds up winning: https://en.wikipedia.org/wiki/First-mover_advantage
Just kidding, kind-of. At least people are jumping in with genuinely useful discussion. Time to delivery of new features winds up winning: https://en.wikipedia.org/wiki/First-mover_advantage
I'll chime in with a comment about minimum viable products.
I have a sense that a lot of MVP's are built on top of a spotty foundation, whether due to a lack of experience or knowledge (Who thinks to implement EF with a pattern like the Repository or Unit of Work patterns the first time they use it? Who architects their first MVC application with layers?) and once that foundation is in place it is hard to find the buy-in, time or budget to improve it. Without a proper foundation, bugs are bound to appear. And then you start trying to Band-Aid the problems in a futile hope they'll go away. (We all know how that story ends.)
A good foundation makes for slower going off the top, but if the MVP is good enough to turn into a real product the effort will definitely pay for itself in the long run. And frankly, a lot of the foundational items (patterns, architecture) that could be baked in from the start can be ported to future efforts (whether a pivot or a new product), so it is worth the time to learn how to implement them early on when investing in a new knowledge stack.
I have a sense that a lot of MVP's are built on top of a spotty foundation, whether due to a lack of experience or knowledge (Who thinks to implement EF with a pattern like the Repository or Unit of Work patterns the first time they use it? Who architects their first MVC application with layers?) and once that foundation is in place it is hard to find the buy-in, time or budget to improve it. Without a proper foundation, bugs are bound to appear. And then you start trying to Band-Aid the problems in a futile hope they'll go away. (We all know how that story ends.)
A good foundation makes for slower going off the top, but if the MVP is good enough to turn into a real product the effort will definitely pay for itself in the long run. And frankly, a lot of the foundational items (patterns, architecture) that could be baked in from the start can be ported to future efforts (whether a pivot or a new product), so it is worth the time to learn how to implement them early on when investing in a new knowledge stack.
Restarting the browser is a valid solution to alot of problem. Web caching is a definite performance requirement for web apps especially when you don't know what sort of bandwidth a user will be using against your site. Alot of users could be running on a budget phone or tablet, running IE 6 as well as the current supported browsers.
So I think picking on Web Apps as a whole is wrong. It depends entirely on the framework. Is it an established framework like ruby on rails or the latest bleeding edge JS framework ?
The issue is development platform specific and possibly the cultures within a development company. Definitely a problem for Games Consoles.
So I think picking on Web Apps as a whole is wrong. It depends entirely on the framework. Is it an established framework like ruby on rails or the latest bleeding edge JS framework ?
The issue is development platform specific and possibly the cultures within a development company. Definitely a problem for Games Consoles.
I have to say continuous delivery may be perceived as a way of shipping features. Perhaps it should be viewed with even higher value as a way to deploy bug fixes.
@Keith, maybe you are talking about Architecture when you say "Thinking clearly". Like those of us with experience perceive the impact on the system as a whole instead of all the little things that get us there. When I say Architecture I don't mean off-the-shelf GOF Design Patterns, but a consistent way of doing things that ensure quality.
Like you expressed 'a philosophy' that perhaps melds your experience of the principal benefits of things like DRY, SOLID, YAGNI and lean and agile. You act as a master of creating software.
One issue I see is that the more inexperienced are the loudest voices. This results in adopting the newest buzz methodology seem like best practice. Especially to business folk that have no experience that informs them of the pitfalls.
Perhaps businesses value the wrong kind of skills. Although during the hiring process how do you evaluate Discipline?
We have come a long way with the tooling. Maybe faster than our thinking has changed to match it. Better tools only makes us better users, it doesn't make us better makers. It still comes back to discipline and what I will call Architectural Intelligence. All the testing in the world wont make you a better programmer or your app more secure. Making the decision to be better will.
Something else I see that fails to produce the desired outcome is thinking that programming or testing is simple math. I hear people say thing that each method should have two unit tests, or if your class is the third generation in inheritance you did something wrong, or if you use interface inheritance you should refactor... Maybe those are great rules for someone who started programming a month ago because they are not ready to know when it is good or bad to do certain things.
The fact of the matter is that TDD doesn't make perfect code. Continuous delivery doesn't make better code. Agile doesn't make better software. Good people make good software. ...and Bug reports from strangers are gifts.
If your project suffers from quality, you should find like-minded people. You will be more satisfied with the software you will create.
If you have enough experience, decided you want to be better, and continuously deliver on that desire you can create software that doesn't make you sad. Even if it is Minimal Viable Product seeking First Mover Advantage as directed by the business.
:)
@Keith, maybe you are talking about Architecture when you say "Thinking clearly". Like those of us with experience perceive the impact on the system as a whole instead of all the little things that get us there. When I say Architecture I don't mean off-the-shelf GOF Design Patterns, but a consistent way of doing things that ensure quality.
Like you expressed 'a philosophy' that perhaps melds your experience of the principal benefits of things like DRY, SOLID, YAGNI and lean and agile. You act as a master of creating software.
One issue I see is that the more inexperienced are the loudest voices. This results in adopting the newest buzz methodology seem like best practice. Especially to business folk that have no experience that informs them of the pitfalls.
Perhaps businesses value the wrong kind of skills. Although during the hiring process how do you evaluate Discipline?
We have come a long way with the tooling. Maybe faster than our thinking has changed to match it. Better tools only makes us better users, it doesn't make us better makers. It still comes back to discipline and what I will call Architectural Intelligence. All the testing in the world wont make you a better programmer or your app more secure. Making the decision to be better will.
Something else I see that fails to produce the desired outcome is thinking that programming or testing is simple math. I hear people say thing that each method should have two unit tests, or if your class is the third generation in inheritance you did something wrong, or if you use interface inheritance you should refactor... Maybe those are great rules for someone who started programming a month ago because they are not ready to know when it is good or bad to do certain things.
The fact of the matter is that TDD doesn't make perfect code. Continuous delivery doesn't make better code. Agile doesn't make better software. Good people make good software. ...and Bug reports from strangers are gifts.
If your project suffers from quality, you should find like-minded people. You will be more satisfied with the software you will create.
If you have enough experience, decided you want to be better, and continuously deliver on that desire you can create software that doesn't make you sad. Even if it is Minimal Viable Product seeking First Mover Advantage as directed by the business.
:)
My biggest pet peeve - developers who preach TDD. Testing is good. But TDD is bad, IMHO.
I remember reading your prior rant and thinking "I'm not alone!". I agreed then and I agree now. We have clearly spent a lot of time considering patterns, practice and process. What isn't quite as clear is how this has improved the end results.
For a science focused so heavily on computation and data we are oddly willing to make lasting decisions based on anecdotal evidence.
For a science focused so heavily on computation and data we are oddly willing to make lasting decisions based on anecdotal evidence.
@Gerry Lowry
Developers need to remember that every five minutes spent coding may generate hours of testing. A bug obvious to the developer may not surface under test.
Sure, everyone in QA can become a developer and deep-dive into the code, but then the developers complain that the testers are "doing too much." (And, of course, the better testers eventually either quit, or get vastly better paying jobs... as developers.)
Developers need to remember that every five minutes spent coding may generate hours of testing. A bug obvious to the developer may not surface under test.
Sure, everyone in QA can become a developer and deep-dive into the code, but then the developers complain that the testers are "doing too much." (And, of course, the better testers eventually either quit, or get vastly better paying jobs... as developers.)
I think our industry requires a watch dog to audit software development practices and report their findings to prospective clients.
Until there's a watchdog to keep software developers professional then this industry will continue to be the Wild West of the economy.
The DotNetRocks show 1214 talks about this culture which makes up the majority of this industry.
Until there's a watchdog to keep software developers professional then this industry will continue to be the Wild West of the economy.
The DotNetRocks show 1214 talks about this culture which makes up the majority of this industry.
- Agile has 50 percent good ideas and 50 percent sales pitch for training courses, consulting coaches, agile mentors, ...
- Frameworks by the bushel web sites, applications directly cause buggy high support cost and limited lifecycle code bases due to weakest link in the chain syndrome. Consider a large web site with 25+ different javascript libraries
- Vendors unwilling to kill off/end of life development tools and technologies directly causes poor quality by giving many parallel ways to solve a given task & allowing applications to have 3+ different generations of tools/libraries/methodologies/frameworks
- Tool/Library vendors introducing a new big-bang technology every 3 years to sell product, courses, books, certification exams, etc. causes poor quality by multiplying the number of tools/libraries/frameworks in a set of solutions. Consider actually making the development tool/library/framework easier to use, better documented and with actual meaningful exception messages
- Development teams picking their pet set of favourite tools, build engine, agile tools, methodologies, coding style guidelines, etc for each new project causes bad quality at the software portfolio level by the proliferation of different points of failure.
- Shipping apps with the business plan of never updating them
- Vendors failing to invest in things lowering the TCO of software, such as an XCOPY deploy with no registry entries, not using COM, not requiring an active network link for essentially an offline piece of software, not forcing updates, ...
- Vendors failing to test that their uninstall completely removes all remnants of the application
- Industry acceptance and slouch towards lowest common denominator development tools and languages little better than 'Fortran of the web' AKA JavaScript. What C# or C++ programmer would write in a system where each class has a property bag for all of its internal data? That was discarded as a bad idea more than 10 years ago, yet it thrives in JavaScript base web site code. Vendors could fast track Polymer based solutions to get the web programming back to par with early 1980s structured programming systems
- IT/CS majors would be much better off to have a 1 hour every semester of Computer History 1930s-1970s.
- Frameworks by the bushel web sites, applications directly cause buggy high support cost and limited lifecycle code bases due to weakest link in the chain syndrome. Consider a large web site with 25+ different javascript libraries
- Vendors unwilling to kill off/end of life development tools and technologies directly causes poor quality by giving many parallel ways to solve a given task & allowing applications to have 3+ different generations of tools/libraries/methodologies/frameworks
- Tool/Library vendors introducing a new big-bang technology every 3 years to sell product, courses, books, certification exams, etc. causes poor quality by multiplying the number of tools/libraries/frameworks in a set of solutions. Consider actually making the development tool/library/framework easier to use, better documented and with actual meaningful exception messages
- Development teams picking their pet set of favourite tools, build engine, agile tools, methodologies, coding style guidelines, etc for each new project causes bad quality at the software portfolio level by the proliferation of different points of failure.
- Shipping apps with the business plan of never updating them
- Vendors failing to invest in things lowering the TCO of software, such as an XCOPY deploy with no registry entries, not using COM, not requiring an active network link for essentially an offline piece of software, not forcing updates, ...
- Vendors failing to test that their uninstall completely removes all remnants of the application
- Industry acceptance and slouch towards lowest common denominator development tools and languages little better than 'Fortran of the web' AKA JavaScript. What C# or C++ programmer would write in a system where each class has a property bag for all of its internal data? That was discarded as a bad idea more than 10 years ago, yet it thrives in JavaScript base web site code. Vendors could fast track Polymer based solutions to get the web programming back to par with early 1980s structured programming systems
- IT/CS majors would be much better off to have a 1 hour every semester of Computer History 1930s-1970s.
I've been doing a lot of development using Xamarin Forms. So, not to pick on Xamarin because they are doing something amazing (i.e. they picked a hard problem) but it has been a real roller coaster fighting the high degree of churn, in both the frameworks and the tooling.
Sometimes I feel like I am paying thousands of dollars a year to be a beta tester.
However,the awesomeness of using shared C# and having it run on both the iOS and Android platforms is intoxicating enough that I am sticking with it, regardless of the speed bumps.
Sometimes I feel like I am paying thousands of dollars a year to be a beta tester.
However,the awesomeness of using shared C# and having it run on both the iOS and Android platforms is intoxicating enough that I am sticking with it, regardless of the speed bumps.
At the beginning of the Windows era, I went to some IT show in Paris. At that time Lotus 123 spreadsheet was THE software people wanted. I went to talk to them about a bug I found. They laughed at me saying bug was not possible as the software was used by millions of people. I couldn't answer anything because bugs were indeed nearly non existant though they were programmed in Assembly Language at least for Lotus even for the Windows version (whereas Excel was in C I think, by the way I compared both and Lotus was much more performant ok not the subject ;) ).
So why did we enter the Big Bang Times of bugs ? Because programming model changed, event programming is no more deterministic though necessary to cope with user interface interactivity. Second more and more people entered the job Market with no real engineering background and that was also necessary due to demand growth. Third toolings are far behind complxity growth. I confess Visual Studio is the best but fundamentally even its code editor has not much evolved. It's amazing programmers have to code so much or use frameworks and servers they do not really understand because documentation suck and they don't have Time to look under the hood. Why did Visual Studio did not improve and even retire Object Workbench instead of just copying BlueJ? Fourth Languages oh yeah I know there seem to be a lot of evolutions functional programming, Roslyn etc. but I doubt it will fundamentally improve the way to create software.
In conclusion programmers continue to code they do not really design i.e. model their software. Problem is tools / languages are designed by academics who are experts of making compilers and uml but not of the way an average person needs to design his application at a higher level of abstraction. That's why Microsoft to throw "M" which Marketing tried to make believe even your mother could model without telling your mother have to get a PHD.
Some will pretend There is no easy way. I would answer there is. In fact Paul Allen had already done it unfortunately he was too ahead of his Time and lack the Marketing genius of Bill Gates. But I know for the story that when Bill Gates saw it he said to Paul Allen: your product is fantastic, I will copy it. So he bought Visual Basic but it has nothing to do with Paul Allen product. I will tell you ils name : Toolbook. In that product you can craft / code complex components easier than with Photoshop and immediately test it. You don't have to cope with obscure Visual Studio DTE APIs to craft your own toolings to be able to be productive and so concentrate on doing well instead of beggering for answers on Stackoverflow.
For this is the root cause of non quality in software industries : not doing things at the right level of abstractions and doing Q/A to compensate is costfull and much too late. As Deming said :
"Inspection does not improve the Quality, nor guarantee Quality. Inspection is too late. The Quality, good or bad, is already in the product.
Eliminate the need for inspection on the mass basis by building Quality into the product in the first place."
Edwards Deming - Out of the Crisis.
So why did we enter the Big Bang Times of bugs ? Because programming model changed, event programming is no more deterministic though necessary to cope with user interface interactivity. Second more and more people entered the job Market with no real engineering background and that was also necessary due to demand growth. Third toolings are far behind complxity growth. I confess Visual Studio is the best but fundamentally even its code editor has not much evolved. It's amazing programmers have to code so much or use frameworks and servers they do not really understand because documentation suck and they don't have Time to look under the hood. Why did Visual Studio did not improve and even retire Object Workbench instead of just copying BlueJ? Fourth Languages oh yeah I know there seem to be a lot of evolutions functional programming, Roslyn etc. but I doubt it will fundamentally improve the way to create software.
In conclusion programmers continue to code they do not really design i.e. model their software. Problem is tools / languages are designed by academics who are experts of making compilers and uml but not of the way an average person needs to design his application at a higher level of abstraction. That's why Microsoft to throw "M" which Marketing tried to make believe even your mother could model without telling your mother have to get a PHD.
Some will pretend There is no easy way. I would answer there is. In fact Paul Allen had already done it unfortunately he was too ahead of his Time and lack the Marketing genius of Bill Gates. But I know for the story that when Bill Gates saw it he said to Paul Allen: your product is fantastic, I will copy it. So he bought Visual Basic but it has nothing to do with Paul Allen product. I will tell you ils name : Toolbook. In that product you can craft / code complex components easier than with Photoshop and immediately test it. You don't have to cope with obscure Visual Studio DTE APIs to craft your own toolings to be able to be productive and so concentrate on doing well instead of beggering for answers on Stackoverflow.
For this is the root cause of non quality in software industries : not doing things at the right level of abstractions and doing Q/A to compensate is costfull and much too late. As Deming said :
"Inspection does not improve the Quality, nor guarantee Quality. Inspection is too late. The Quality, good or bad, is already in the product.
Eliminate the need for inspection on the mass basis by building Quality into the product in the first place."
Edwards Deming - Out of the Crisis.
Happening? Oh man, is it happening. Just a case in point - check out the "new" Android Pluralsight app. Broken in too many ways to enumerate here and they are "fixing" it by releasing updates on a near daily basis.
Stupid decisions were made just in the design - things like forgetting that some Android devices are tablets and that some things display better in landscape then portrait. Things like forgetting that there are not one, but two "Ups" (spin it 180 degrees when starting in portrait mode and it's still "up" in portrait mode).
The old app used to do that, the new one...nope.
So...now we, the *paying* public are not only QA testing, bug finding victims, but we are now forced to help them design their UI by screaming at them for doing blatantly stupid things.
And, for "Anon" above, who stated:
"Developers need to remember that every five minutes spent coding may generate hours of testing. A bug obvious to the developer may not surface under test.
Sure, everyone in QA can become a developer and deep-dive into the code, but then the developers complain that the testers are "doing too much." (And, of course, the better testers eventually either quit, or get vastly better paying jobs... as developers.)"
This is THE reason that UNIT TESTS were invented. Don't spend that five minutes writing code, spend a five minutes with the code and another 10 writing some comprehensive unit tests and see how well things go after that. (Yeah, yeah, I know, preaching to the choir here, but sheesh!). And when I say "unit test" I, of course, mean any appropriate test; unit, integration, etc. Whatevern is appropriate.
Ensure that the QA people are doing just that - Quality Assurance, as in, did we indeed hook up all the buttons to the correct event handlers, etc. Does the app flow like it's supposed to, etc. I shouldn't, as a user, be running into some of the massively stupid things I've encountered over time (another case in point, said Pluralsight app can't even keep track of which module it's in, skip forward and it skips back again after finishing the subsection you were listening to).
Stupid decisions were made just in the design - things like forgetting that some Android devices are tablets and that some things display better in landscape then portrait. Things like forgetting that there are not one, but two "Ups" (spin it 180 degrees when starting in portrait mode and it's still "up" in portrait mode).
The old app used to do that, the new one...nope.
So...now we, the *paying* public are not only QA testing, bug finding victims, but we are now forced to help them design their UI by screaming at them for doing blatantly stupid things.
And, for "Anon" above, who stated:
"Developers need to remember that every five minutes spent coding may generate hours of testing. A bug obvious to the developer may not surface under test.
Sure, everyone in QA can become a developer and deep-dive into the code, but then the developers complain that the testers are "doing too much." (And, of course, the better testers eventually either quit, or get vastly better paying jobs... as developers.)"
This is THE reason that UNIT TESTS were invented. Don't spend that five minutes writing code, spend a five minutes with the code and another 10 writing some comprehensive unit tests and see how well things go after that. (Yeah, yeah, I know, preaching to the choir here, but sheesh!). And when I say "unit test" I, of course, mean any appropriate test; unit, integration, etc. Whatevern is appropriate.
Ensure that the QA people are doing just that - Quality Assurance, as in, did we indeed hook up all the buttons to the correct event handlers, etc. Does the app flow like it's supposed to, etc. I shouldn't, as a user, be running into some of the massively stupid things I've encountered over time (another case in point, said Pluralsight app can't even keep track of which module it's in, skip forward and it skips back again after finishing the subsection you were listening to).
Spot on Scott.
What bugs me more than bugs themselves are:
1) companies that make it difficult to report bugs
2) companies that don't acknowledge bug reports
3) companies that say. "It works ok for thousands of customers"
What bugs me more than bugs themselves are:
1) companies that make it difficult to report bugs
2) companies that don't acknowledge bug reports
3) companies that say. "It works ok for thousands of customers"
I agree. It is especially annoying on my phone when the spamware apps seem to be trying to update themselves on an almost weekly cycle. I do have a suggestion that may change things. When Don Knuth wrote the initial volumes of the Art of Computer Programming, he offered a bounty of $2.56 to any reader who discovered a bug (the bounty was awarded to the first reader to find the bug). If software companies were forced to pay a bounty for each bug (maybe as a donation to a foundation to promote STEM education), I bet we would see an immediate improvement in the quality of released software.
Wow. Having been around computers since the 1970s (yes, the field isn't necessarily populated by replicants who get retired by Blade Runners at 35), I've concluded that there are a few fundamental facts propelling the problem.
1) the fundamental change to development happened in 1969 when Codd presented his first paper on RDBMS at IBM; the later 1970 paper brought the RM, and thence SQL (but not of Codd), to application development
2) the SQL database (not quite the RM datastore Codd wanted, but close) meant that logic would be centralized with data, thus eliminating gobs and gobs of code
3) COBOL coders of the day resisted, happy to continue with VSAM/CICS and gobs and gobs of code
4) along came The Web, the disconnected client, html, javascript, java, and all the rest
5) the result is that most new graduates for the last decade and a half, at least, view the world as their grandpappies did, one with files and client-centric control of data. Centralized control of data? Satan incarnate.
6) since the world is now defined by client-centric (and unique to each application) code, much more code is needed to control the data; this more code rapidly becomes unwieldy, and coders now write code in the debugger
7) in the olde days, coders obeyed the ancient carpenter's stricture: measure twice, cut once. Spolsky used to carry the torch for Big Design Up Front; not sure that even he bothers any more. Scrum, et al, dispense with measuring even once.
8) finally, the likes of Google have foisted the notion of Big Data as the one-size-fits-all paradigm, and thus the file oriented (and, thus client-controlled), data paradigm. SQL?? No sql for In Crowd.
This is all unfortunate. With Xeon, SSD, high bandwidth, fully connected web there isn't much need to view the world as disconnected and client-centric. There just isn't. But adopting a centralized, data-centric world means we don't need gobs and gobs of code. We'd need a fraction number of coders. Just as their COBOL grandpappies cried, "I don't know nuthin bout birthin babies!!!", so too today's web coders discard the RM as irrelevant. We could have a development semantic which looks a lot like a *nix RDBMS talking RS-232 to VT-220s. No muss, no fuss. All coders are needed for is to write screens and input routines; the database takes care of the rest. Scary thought, is it not?
Motive and incentive propel human decision making far more than data.
I find it just a tad ironic to read about MongoDB and the semi-fiasco surrounding implementation and cost of $lookup. May be joins on the server aren't such a bad idea after all? I wonder whether the MongoDB folks will claim that $lookup is their brilliant invention? How long before the likes of MongoDB start taking ACID? Ya think?
1) the fundamental change to development happened in 1969 when Codd presented his first paper on RDBMS at IBM; the later 1970 paper brought the RM, and thence SQL (but not of Codd), to application development
2) the SQL database (not quite the RM datastore Codd wanted, but close) meant that logic would be centralized with data, thus eliminating gobs and gobs of code
3) COBOL coders of the day resisted, happy to continue with VSAM/CICS and gobs and gobs of code
4) along came The Web, the disconnected client, html, javascript, java, and all the rest
5) the result is that most new graduates for the last decade and a half, at least, view the world as their grandpappies did, one with files and client-centric control of data. Centralized control of data? Satan incarnate.
6) since the world is now defined by client-centric (and unique to each application) code, much more code is needed to control the data; this more code rapidly becomes unwieldy, and coders now write code in the debugger
7) in the olde days, coders obeyed the ancient carpenter's stricture: measure twice, cut once. Spolsky used to carry the torch for Big Design Up Front; not sure that even he bothers any more. Scrum, et al, dispense with measuring even once.
8) finally, the likes of Google have foisted the notion of Big Data as the one-size-fits-all paradigm, and thus the file oriented (and, thus client-controlled), data paradigm. SQL?? No sql for In Crowd.
This is all unfortunate. With Xeon, SSD, high bandwidth, fully connected web there isn't much need to view the world as disconnected and client-centric. There just isn't. But adopting a centralized, data-centric world means we don't need gobs and gobs of code. We'd need a fraction number of coders. Just as their COBOL grandpappies cried, "I don't know nuthin bout birthin babies!!!", so too today's web coders discard the RM as irrelevant. We could have a development semantic which looks a lot like a *nix RDBMS talking RS-232 to VT-220s. No muss, no fuss. All coders are needed for is to write screens and input routines; the database takes care of the rest. Scary thought, is it not?
Motive and incentive propel human decision making far more than data.
I find it just a tad ironic to read about MongoDB and the semi-fiasco surrounding implementation and cost of $lookup. May be joins on the server aren't such a bad idea after all? I wonder whether the MongoDB folks will claim that $lookup is their brilliant invention? How long before the likes of MongoDB start taking ACID? Ya think?
We used to rotate our software engineers into customer support on a regular basis.
They didn't like it, but when they had to talk to the users directly, over the phone, and listen to how much grief they caused with their bugs or unimplemented feature requests, they got the message.
It had a remarkable and lasting effect on the quality and speed of coding, fixes and enhancements.
I suppose that doesn't scale well these days.
Out with the tried and true and in with the bold and new. 😉
They didn't like it, but when they had to talk to the users directly, over the phone, and listen to how much grief they caused with their bugs or unimplemented feature requests, they got the message.
It had a remarkable and lasting effect on the quality and speed of coding, fixes and enhancements.
I suppose that doesn't scale well these days.
Out with the tried and true and in with the bold and new. 😉
The math/science has been there for 150 years - think of observability, controllability and feedback loops in system control. Software shouldn't be any different... should be 'engineered'.
Developing a software system only makes things easier, in all of the stages in the product lifecycle - development, deployment, maintenance and retirement.
I personally think the development methodology is of secondary importance. The shared sense of responsibility for the product comes first. This would be the right mindset.
With the right mindset, the 'good enough' would really be good enough - no matter you are working on a super sleek app, game or enterprise service/platform.
My dream is to work on a self (or a bit assisted) monitoring, optimizing and correcting system. So you wouldn't even need to wait for users to report an issue... In reality, I'm ok with cutting some corners if it works for the users as well :D
BTW: I'm a big fan of frequent updates and continuous deployment techniques.
Developing a software system only makes things easier, in all of the stages in the product lifecycle - development, deployment, maintenance and retirement.
I personally think the development methodology is of secondary importance. The shared sense of responsibility for the product comes first. This would be the right mindset.
With the right mindset, the 'good enough' would really be good enough - no matter you are working on a super sleek app, game or enterprise service/platform.
My dream is to work on a self (or a bit assisted) monitoring, optimizing and correcting system. So you wouldn't even need to wait for users to report an issue... In reality, I'm ok with cutting some corners if it works for the users as well :D
BTW: I'm a big fan of frequent updates and continuous deployment techniques.
This is true in the video game industry even. I never thought you could pay so much money for something that isn't even finished. Its not like you go to a movie and watch bad takes while they're slowly replacing it. For software, I would expect a "finished" product for the current feature set. With automated testing to much easier to do there really isn't any excuses why all these bugs are being pushed out to the consumer.
As long as the app is usable and customers are happy no need to dig after every tiny bug . I see it as maturity in thinking in software development world .
Well yes and no.
Yes, it is happening. But the overall quality, productivity, and ease-of-use of software today is aeons ahead of what it was 20 - 30 years ago.
So yes the internet makes it too easy to push-bugs-and-update-later, but frankly I prefer to live in a world where this is at least possible. As a consumer I'm happier living with a few minor bugs which are quickly quashed the next day, then to live with one mighty bug for which I have to wait 6 months down the horizon for a fix.
Some app developers will definitely misuse it, but there is just so much choice in software today that anyone consistently misusing the approach will be dropped by the customers as someone better comes along.
Might be more of a problem for huge platforms, and browsers as some comments note, but for the average app in the app store, I'm pretty OK with multiple updates, new features - even experimental ones - and quick fixes.
Yes, it is happening. But the overall quality, productivity, and ease-of-use of software today is aeons ahead of what it was 20 - 30 years ago.
So yes the internet makes it too easy to push-bugs-and-update-later, but frankly I prefer to live in a world where this is at least possible. As a consumer I'm happier living with a few minor bugs which are quickly quashed the next day, then to live with one mighty bug for which I have to wait 6 months down the horizon for a fix.
Some app developers will definitely misuse it, but there is just so much choice in software today that anyone consistently misusing the approach will be dropped by the customers as someone better comes along.
Might be more of a problem for huge platforms, and browsers as some comments note, but for the average app in the app store, I'm pretty OK with multiple updates, new features - even experimental ones - and quick fixes.
Scott,
How can I pre-report a bug against VS 2015 + the open source libraries liberally sprinkled into my solution for 3 years from now when several of the open source libraries or VS handling of them are broken?
@Robert, thanks for the MongoDB and $lookup fiasco information...
Now, back to my daytime project of fixing someone else using cursors in C# to do a foreach over relational data brought back to C# via ORM - not bad design software - good original design but ORM hammer failed to suit future business need to integrate with external systems.
How can I pre-report a bug against VS 2015 + the open source libraries liberally sprinkled into my solution for 3 years from now when several of the open source libraries or VS handling of them are broken?
@Robert, thanks for the MongoDB and $lookup fiasco information...
Now, back to my daytime project of fixing someone else using cursors in C# to do a foreach over relational data brought back to C# via ORM - not bad design software - good original design but ORM hammer failed to suit future business need to integrate with external systems.
Even Yahoo is getting out of Software Quality. I guess it's time to change professions. ieee.org
Comments are closed.
At one point, Chrome introduces a severe regression, that most people didn't even notice yet it was severe: you couldn't fast seek an audio file. A fast seek would cause the browser to fetch all bytes up to that point. Potential increased bandwidth cost for users, downgraded experience, increased bandwidth costs for site owners. That's the ticket if you're interested: https://code.google.com/p/chromium/issues/detail?id=530043
Eventually the Chromium team (the bug was in Chromium itself) resolved it after some tight help from us, but due to their release cycle, the fix went out only a month or so later.
So: Chrome is really "evergreen", which is often good, but sometimes apparently introduces some serious breaking changes or bugs.
I wrote in a blog post ( http://fullstack.info/increased-bandwidth-consumption-and-cost-all-over-the-internet-due-to-chrome-45-regression/ ):
"I understand that being on the bleeding edge (“help move the web forward” is in Chromium’s slogan) is prime, and that’s the cost we pay, as the alternative would be the known Internet Explorer saga. But we should also urge Google and the Chromium + Chrome teams to fix this issue, which has severe experience and monetary implications on everyone."