Showing posts with label software development. Show all posts
Showing posts with label software development. Show all posts

Monday, January 13, 2014

Is Windows a good development platform?


    I've had a conversation with my colleague couple days ago about is Windows a good platform for development or not. I was frustrate with some of MS software at that point, I was arguing that even though developers are the power of Windows as a platform, they were never treated as good, as in Linux (or Mac now).
    On one hand, Windows has arguably best in class IDE for C/C++, C#, Visual Basic - Visual Studio. Note, it's only three out of dozens of main stream languages. And by been closed to external expansion - they didn't implement system like Eclipse, where others can add support for other languages. Now, is that would add a lot of work for MS developers - not really, if they architected VS in the right way - it's already pluggable system (they have number of different languages, with different syntaxes - it just makes sense to have abstract system, where each language is an add-on), so it's just a matter of sharing this API (and yes, supporting it in next versions).
    Having different programming languages in Visual Studio brings me to next step - you project configuration and build scripts. Each project is not just a set of files with code, but also what compiler to use, flags for the compiler and linkers in different configurations, external libraries and resources required and etc. Now, Visual Studio organizes all this in XML file, which presumably is human readable and editable (XML is not a good format for this anyway - too much text is just too much text). But compare a 4 configuration setup, when in that XML you will have 4 sections of pretty much the same text, except may be couple flags absent or present and CMake, where it's organized as common block and if blocks with specifics of each configuration. Now when you want to edit only release - you will need to go through two same blocks and find what you need in a pile of options, or find appropriate if RELEASE and change options there. This continues with external libraries and etc (it's really pain adding an library even from Visual Studio, because you need to remember to select all configurations, as if adding library to only one is a default case everybody needs).
    Ok, for example, we got our C++ project up and running and checked into repository. Called up a colleague and asked to add a feature to it... Oh wait, they just got a new machine and need to install environment. Can we help him somehow? Not really, only give bunch of links to download ton of software and wait until they click through hundreds of wizards. Note, that we just need a (clean) clone of our environment on different machine, but there is really (except using VMs) no way to just give him a setup script, that would install everything automatically. Hey, that's a day he will not be able to return to his life.
     So our colleague installed everything, but our project doesn't compile on his machine. He looks and finds out that he installed third party library of incorrect version and to incorrect path. Because it actually asked where to install it - why there is no specific place, where libraries are installed? And yes, we didn't specify version, because how would we do it - only add version number to the folder it's installed, again manual and pretty complicated to keep track and update later. Simple install script, that would install required versions would save time and make sure of appropriate version and place. Now we institute a special place for libraries to be put or even include them in repository to make sure versions are the same for everybody and go on, but feeling that this OS doesn't know what any developer will need still left.
     Switching gears, we were asked to write a web scrapper. Of cause you can try to do it in C#, but that will be overkill - number of lines of code you will write is just insane compared to scripted languages. So our choices are Perl and Python. We go with Python - and again everything starts - download Python, install by clicking on a wizard. This time no Visual Studio to back you up - so open Notepad... oh, wait, the default editor is just nightmare. How for the love of god default editor can be that bad? Anyway, download Notepad++ (again wizard, click-click, time is passing by). Python has great system to install required libraries, which in Linux/Mac works like a charm. It works in Windows, until you need to compile some C/C++ code for it. How is it possible that no one man figured out how to compile programmatically their code when Visual Studio installed from Python. It works well with GCC, but I've tried multiple times, including figuring out code that does compilation with Visual Studio and failed to configure it right. No, there is precompiled packages you can download and install (wizard, click-click) and from what I understand they are compiling them with MinGW (prove me wrong).
    Client who asked about web scrapper - asked to save everything to MySQL database. We need to install it (you know it - wizard, click-click), we need to configure it (UTF8 by default, different port and etc) - quest to find where configuration file is, because who looked at where that mysql asked us to install itself. Ok, now how is our colleague or where we deploy our code will have same environment? Again, write a document where to download everything, and attach a config file to that document.
     Concluding, both *nix and Windows platforms have their positive and negative sides. Linux has a lot of development awareness just because developers is a huge user base, and any developer can have a say at how it should be. I wanted to point out that Windows, as OS should be much more aware of this issues and open to help developers to do their job better and easier, and in result spread MS products. Instead, by been not really helpful in a lot of this details it repeals them. Now days, Mac OS by been easy to use from one hand and actually having a lot of Linux flexibility (*nix file system, bash scripting, MAC ports and Homebrew to install packages) grows as a primary platform where development is been made. Of cause another trend is developing web application, when you just connect to a server and write code there. Windows by having only graphics remote desktop requires much higher bandwidth, which not always available. In result, it's Linux who shines, by providing powerful text-based IDE in addition to great development environment. 

Friday, December 28, 2012

Release cycle - Part II

      This is additional part to previous post about release cycle.

      So I continued to read +Joel Spolsky and guess what? He actually ended up with Kanban ideas in mind in one of his latest posts. Apparently I should read all Joel's stuff before commenting :)

      Basic idea behind "pull"-kind of scheduling system (as I understand it), is getting features to customer as fast as it possible. Now, saying this - your release cycle should be getting faster and faster. And for this, of cause, you will need all this modern tools (TDD and unit-tests, DVCS and etc), which will allow to push you changes, that are ready for deployment, to production server and know that feature works and has high quality.

      By making "pull" system, you'll illuminate most of "waste" inventory and will be able to show customers same features you use "in-house". Imagine, you have configured you Continues what-ever system (Delivery or Deployment) - system that allows to get code that works through test-system to the production (web service or auto-update server). But this is only part of job for "pull" system.

      You also need to have police to choose features to develop only based on "highest" priority. This means, that if you have a mile long backlog, you will need to sort it by priority (number of clients asking for feature and\or $$ additionally gained) and acknowledge to yourself, that low priority features won't be done... ever. You'll always get new ideas for features from clients or from team - that will go with high priority. So don't spend time to revise them again - throw them out. If any of that features was really important - it will come up again.

     Ok, that's all fun when you have perfect code, it's all separated by modules and it's all covered by unit-tests and etc. But let's look truth in the eyes - code is a mess and tests are inferior. And this means that refactoring is required, while we want to push new features out (customers are waiting and nobody wants to disappoint them).

     To get refactoring going - let's make in a backlog a feature with concreate proposition how to refactor parts of code (not just - "rework this", but concreate "this should have interface A, B, implement algorithm C") and set a high priority to it. Then when current features are done, one by one refactoring tasks will be performed by developers. Proposition for refactoring should have description how new code should be tested (unit-tests, integration-tests, system-tests, performance-tests) to ensure that it works correctly. If new high priority feature gets into play - it will be after some of refactoring tasks, that already got in the queues of developers.

    While code is still imperfect - release cycle will be maintained not continues, but it should be minimized. From years to month as first steps. When new minor version will be released each month - next step go to weeks - just a couple of features (sometimes small and sometimes rather large). While this - refactoring should be going to get code to better shape and unlock possibility to deliver software continuously.

Wednesday, December 26, 2012

Release cycle

      And again let's start the topic from +Joel Spolsky post about "Picking a Ship Date". His main idea in the article is:
  • If product is brand new - release often
  • If product have maturity - 1 year - 2 years release cycle is for you
  • If product is platform - at least 3 years.
      Ok, the article is from 2002 (what? yep, that old). So there was no distributed version control system (no, really, first releases of Git or Mercurial were at April, 2005). 

      Now, how did it changed the world? It actually changed it pretty dramatically. Before the main workflow was to develop a set of big features and then go to "pre-release cycle", when you fix issues, add small features and get your QA cracking your software. Because of that if you have 1 year release cycle, you would have only 3-5 month to get big features to the system, and then you have this cycle of getting software to production form.

      Behold, DVCS gave you an option to have master branch in "release" form all the time. If you are doing a work that requires more than one commit (which any normal work does - because you should commit frequently) you just put in separate branch, which will be merged when it's ready (and may be even tested by QA). When in old world you would just make you current state unreleasable by pushing your commit to trunk (ok, there was branches in SVN... but really, did somebody used them to make new features?).

     Let's look at Github - they've made 2000+ releases at 6 month period. Some of them sure were pretty small - a bug fix, a simple button that million customers asked to add or just a tooltip that made better usability. But some of them were pretty big - new interface, or changing backend, or may be new system of distributed handling of repositories (I'm making this up, though). The idea is - each release were the same - somebody finished his work in separate branch and merged it to main (release) branch.

    Another example is Google Chrome - I think it's the best update system in the world that is in this world. IT DOESN'T NEED ME TO DO ANYTHING (I'm looking at you Flash, Java, ...)! Yes, and I like it. And they managed over past 4 years I'm using it - no to change interface unexpectedly - so the argument of "too frequent releases will affect usability" doesn't sound if the development process is built with usability in mind.

     As a conclusion I would say, that frequent releases are a good thing when update system is very good (web service or auto-update without person even noticing) and when your have very concreate plan of features that will be implemented and not break usability (features add functionality, not complexity of UI). 

Sunday, October 21, 2012

Developemnt environment - Part I


Development environment - Part I

The objective of this series of posts, is to figure out an environment for developing cross-platform multi-targeted (desktop and web, SaaS, front-end and back-end parts) and multi-lingual (C++, Python, Java, JS) applications using TDD as quality control and Continues Delivery to release faster and more frequently. This post is mostly for my own reference, but you may find it useful as well (or comment on what I did wrong :) ). I hope that this series of posts will initiate a discussion on this topic, where everybody will participate to figure out best layout.

In this first post, I want to describe requirements in deep, and then step by step in next posts build a system that will satisfy them.

So, let’s again describe requirements, that environment should satisfy:
1) Software will be cross-platform: Windows, Linux, FreeBSD, Mac OS, HP-UX, AIX, iOS, Android.
This also adds additional plane for thinking - people want to use best IDEs that are available in concrete platforms. For example for development in C++ at Windows - Visual Studio will be the best choice, and developers wouldn’t want to downshift to Eclispe or other cross-platform, but less developer-friendly systems.
2) Software will be multi-targeted: application will have back-end and front-end sides, each of which can be run on multiple platforms - back-end may be running on desktop, server or cloud, when different front-ends will be running on desktops and in the web.
3) Multi-lingual: Lowest level of back-end will be running on C++ to achieve best efficiency, when higher levels of back-end may run on Python (faster development, less code). Front-end may be containing Java and\or JS (desktop and web).
4) System should be extensively tested on regular bases (best if after each commit) by unit-tests, integration tests, performance tests.
5) System should provide Continues Delivery, that will allow to put a feature to production at the same day as it was finished (passed all tests as well) and allow users give a feedback about it. But if one of new features did brake build - this shouldn’t stop other new features from be released.
6) Environment should have outline behaviour for teams of different size and on different stage of development. (Example, at the beginning when couple people may work on same component, and they should work in the same branch, on the other hand at feature-creep stage - each person works on separate small\medium size feature and should work in own branch).

If we go for more detailed numbering of IDEs on each platform:

  • Windows: 
    • Visual Studio for C++;
    • Notepad++ for Python, JS;
    • Vim for C++, Java, Python, JS;
    • Eclipse for C++, Java, Python, JS;
    • QtCreator for C++.
  • Linux: 
    • Vim for C++, Java, Python, JS; 
    • Eclipse for C++, Java, Python, JS;
    • QtCreator for C++.
  • Mac OS: 
    • Vim for C++, Python, JS; 
    • Eclipse for C++, Java, Python, JS; 
    • XCode for C++;
    • QtCreator for C++

As we see, Eclipse is pretty universal IDE, as well as vim. But developers are different in nature, and some want to get the best tools and development experience money can buy on each platform - i.e Visual Studio for Windows, XCode for Mac OS. Others, will tolerate “common” denominator and would want to work in universal development environment. Some, will enjoy using QtCreator which is cross-platform and pretty good for C++.
In result, we want our solutions\projects to be generated from single source to multiple destinations (VS projects, Eclipise projects, and support QtCreator) and be able to work with plain make system as well.

Quality of build is very important especially if Continues Delivery required. To ensure that the fresh build is qualitative, next activities should be performed:


  • Static code analysis to straighten out code and prevent issues.
  • Unit tests (each piece of functionality should be tested; should be fast and not involve any external components or services (no filesystems, databases etc))
  • Component tests (each component should be tested as combination of units)
  • System tests (multiple comonents working together)
  • Functional tests (testing functionality from user’s perspective\user stories)
  • Load and performance tests (to ensure speed and stability)
  • Issues and bugs should be translated to appropriate type of tests.
  • Code-coverage to ensure that functionality is tested
  • Deployment to staging environment, where system can be tested by QA or by subset of customers.


List of performed activities can be increased if needed. As well as requirements will be more detailed over time.
If somebody see that I missed something - please leave a comment.

UPD. QtCreator added from Vlad's comment.