Release cycle - Part II

      This is additional part to previous post about release cycle.

      So I continued to read +Joel Spolsky and guess what? He actually ended up with Kanban ideas in mind in one of his latest posts. Apparently I should read all Joel's stuff before commenting :)

      Basic idea behind "pull"-kind of scheduling system (as I understand it), is getting features to customer as fast as it possible. Now, saying this - your release cycle should be getting faster and faster. And for this, of cause, you will need all this modern tools (TDD and unit-tests, DVCS and etc), which will allow to push you changes, that are ready for deployment, to production server and know that feature works and has high quality.

      By making "pull" system, you'll illuminate most of "waste" inventory and will be able to show customers same features you use "in-house". Imagine, you have configured you Continues what-ever system (Delivery or Deployment) - system that allows to get code that works through test-system to the production (web service or auto-update server). But this is only part of job for "pull" system.

      You also need to have police to choose features to develop only based on "highest" priority. This means, that if you have a mile long backlog, you will need to sort it by priority (number of clients asking for feature and\or $$ additionally gained) and acknowledge to yourself, that low priority features won't be done... ever. You'll always get new ideas for features from clients or from team - that will go with high priority. So don't spend time to revise them again - throw them out. If any of that features was really important - it will come up again.

     Ok, that's all fun when you have perfect code, it's all separated by modules and it's all covered by unit-tests and etc. But let's look truth in the eyes - code is a mess and tests are inferior. And this means that refactoring is required, while we want to push new features out (customers are waiting and nobody wants to disappoint them).

     To get refactoring going - let's make in a backlog a feature with concreate proposition how to refactor parts of code (not just - "rework this", but concreate "this should have interface A, B, implement algorithm C") and set a high priority to it. Then when current features are done, one by one refactoring tasks will be performed by developers. Proposition for refactoring should have description how new code should be tested (unit-tests, integration-tests, system-tests, performance-tests) to ensure that it works correctly. If new high priority feature gets into play - it will be after some of refactoring tasks, that already got in the queues of developers.

    While code is still imperfect - release cycle will be maintained not continues, but it should be minimized. From years to month as first steps. When new minor version will be released each month - next step go to weeks - just a couple of features (sometimes small and sometimes rather large). While this - refactoring should be going to get code to better shape and unlock possibility to deliver software continuously.

Release cycle

      And again let's start the topic from +Joel Spolsky post about "Picking a Ship Date". His main idea in the article is:

• If product is brand new - release often
• If product have maturity - 1 year - 2 years release cycle is for you
• If product is platform - at least 3 years.

      Ok, the article is from 2002 (what? yep, that old). So there was no distributed version control system (no, really, first releases of Git or Mercurial were at April, 2005). 

      Now, how did it changed the world? It actually changed it pretty dramatically. Before the main workflow was to develop a set of big features and then go to "pre-release cycle", when you fix issues, add small features and get your QA cracking your software. Because of that if you have 1 year release cycle, you would have only 3-5 month to get big features to the system, and then you have this cycle of getting software to production form.

      Behold, DVCS gave you an option to have master branch in "release" form all the time. If you are doing a work that requires more than one commit (which any normal work does - because you should commit frequently) you just put in separate branch, which will be merged when it's ready (and may be even tested by QA). When in old world you would just make you current state unreleasable by pushing your commit to trunk (ok, there was branches in SVN... but really, did somebody used them to make new features?).

     Let's look at Github - they've made 2000+ releases at 6 month period. Some of them sure were pretty small - a bug fix, a simple button that million customers asked to add or just a tooltip that made better usability. But some of them were pretty big - new interface, or changing backend, or may be new system of distributed handling of repositories (I'm making this up, though). The idea is - each release were the same - somebody finished his work in separate branch and merged it to main (release) branch.

    Another example is Google Chrome - I think it's the best update system in the world that is in this world. IT DOESN'T NEED ME TO DO ANYTHING (I'm looking at you Flash, Java, ...)! Yes, and I like it. And they managed over past 4 years I'm using it - no to change interface unexpectedly - so the argument of "too frequent releases will affect usability" doesn't sound if the development process is built with usability in mind.

     As a conclusion I would say, that frequent releases are a good thing when update system is very good (web service or auto-update without person even noticing) and when your have very concreate plan of features that will be implemented and not break usability (features add functionality, not complexity of UI). 

Developemnt environment - Part I

Development environment - Part I

The objective of this series of posts, is to figure out an environment for developing cross-platform multi-targeted (desktop and web, SaaS, front-end and back-end parts) and multi-lingual (C++, Python, Java, JS) applications using TDD as quality control and Continues Delivery to release faster and more frequently. This post is mostly for my own reference, but you may find it useful as well (or comment on what I did wrong :) ). I hope that this series of posts will initiate a discussion on this topic, where everybody will participate to figure out best layout.

In this first post, I want to describe requirements in deep, and then step by step in next posts build a system that will satisfy them.

So, let’s again describe requirements, that environment should satisfy:
1) Software will be cross-platform: Windows, Linux, FreeBSD, Mac OS, HP-UX, AIX, iOS, Android.
This also adds additional plane for thinking - people want to use best IDEs that are available in concrete platforms. For example for development in C++ at Windows - Visual Studio will be the best choice, and developers wouldn’t want to downshift to Eclispe or other cross-platform, but less developer-friendly systems.
2) Software will be multi-targeted: application will have back-end and front-end sides, each of which can be run on multiple platforms - back-end may be running on desktop, server or cloud, when different front-ends will be running on desktops and in the web.
3) Multi-lingual: Lowest level of back-end will be running on C++ to achieve best efficiency, when higher levels of back-end may run on Python (faster development, less code). Front-end may be containing Java and\or JS (desktop and web).
4) System should be extensively tested on regular bases (best if after each commit) by unit-tests, integration tests, performance tests.
5) System should provide Continues Delivery, that will allow to put a feature to production at the same day as it was finished (passed all tests as well) and allow users give a feedback about it. But if one of new features did brake build - this shouldn’t stop other new features from be released.
6) Environment should have outline behaviour for teams of different size and on different stage of development. (Example, at the beginning when couple people may work on same component, and they should work in the same branch, on the other hand at feature-creep stage - each person works on separate small\medium size feature and should work in own branch).

If we go for more detailed numbering of IDEs on each platform:

• Windows: 
• Visual Studio for C++;
• Notepad++ for Python, JS;
• Vim for C++, Java, Python, JS;
• Eclipse for C++, Java, Python, JS;
• QtCreator for C++.
• Linux: 
• Vim for C++, Java, Python, JS; 
• Eclipse for C++, Java, Python, JS;
• QtCreator for C++.
• Mac OS: 
• Vim for C++, Python, JS; 
• Eclipse for C++, Java, Python, JS; 
• XCode for C++;
• QtCreator for C++

As we see, Eclipse is pretty universal IDE, as well as vim. But developers are different in nature, and some want to get the best tools and development experience money can buy on each platform - i.e Visual Studio for Windows, XCode for Mac OS. Others, will tolerate “common” denominator and would want to work in universal development environment. Some, will enjoy using QtCreator which is cross-platform and pretty good for C++.
In result, we want our solutions\projects to be generated from single source to multiple destinations (VS projects, Eclipise projects, and support QtCreator) and be able to work with plain make system as well.

Quality of build is very important especially if Continues Delivery required. To ensure that the fresh build is qualitative, next activities should be performed:

• Static code analysis to straighten out code and prevent issues.
• Unit tests (each piece of functionality should be tested; should be fast and not involve any external components or services (no filesystems, databases etc))
• Component tests (each component should be tested as combination of units)
• System tests (multiple comonents working together)
• Functional tests (testing functionality from user’s perspective\user stories)
• Load and performance tests (to ensure speed and stability)
• Issues and bugs should be translated to appropriate type of tests.
• Code-coverage to ensure that functionality is tested
• Deployment to staging environment, where system can be tested by QA or by subset of customers.

List of performed activities can be increased if needed. As well as requirements will be more detailed over time.

If somebody see that I missed something - please leave a comment.

UPD. QtCreator added from Vlad's comment.