Reflections on 2010

Well 2010 has come and soon will be gone, so time to look back.

I have been married for 2 years now although there are ups and downs but I am happy that I have a nice wife and a cute (and she continues to amaze me everyday) daughter.

Being a father does entail some sacrifices, no longer will you have "your" time, but rather it will be the "family's" time or more accurately the "baby's" time. But the smile of Chloe when she see you wake up and when she is scared and runs to hugs you ( only when the mum in law not around) makes it all worth while. But as for having another one .... lets wait and see :)

Hdb finally decides to give me my house, and it is in the process of renovating, left the painting and the carpentry to be done and it should be ready to move in.

Work wise has been good, finally saw a project from the start to the OSAT phase, couldn't see it all the way to the end though, got rope into another project, as a Software Develop ment Manager, its sort of like a Project Lead kinda thing. Challenging though because the project had already been running for sometime before I came in and there are lots of stuff that I need to pick up and be very familiar with. I am aware of the Peter Principle and the Dilbert Principle, hope that this is not the end yet :)

On more personal note, I have been trying to pick up the Diatonic Harmonica, practicing while waiting for my wife to go off work. Always sort of regretted that I never did pick up a musical instr ument, haha now trying to catch up. Seems like a lot of people around my age are playing catch up in music, I know a few who are trying to pick up the piano, trumpet and the violin. 

Hope to play like this guy below someday!

or even this guy!

Till the next time...

Winston W Royce - Father of waterfall

 Uncle Bob posted an article, What killed waterfall could kill agile.http://cleancoder.posterous.com/what-killed-waterfall-could-kill-agile. He mentioned about a person named Dr Winston W Royce (whom uncle bob called the "Father of waterfall") who published a paper,  Managing the Development of Large Software Systems: Concepts and Techniques, in which a system of developement now know as the Waterfall model was described.

 What shocked me in that paper was not that he advocate the waterfall model, but rather he condemns it. In his own words
"I believe in this concept, but the implementation described above is risky and invites failure".

 The funny thing is so many generations of programmers who studied in colleges and university were brought up believing that it was one of (or depending on how old you are) or "the" model of how software actually gets built. Now we know better, that's why we have things like the Agile movement, Kanban and so on

 If you read the paper, you will discover he has views on how to improve the process and what is surprising is how similar it is to practices that we are advocating today.

His 4th and 5th recommendation looked remarkably like Agile practices.

 Step 4: Plan, Control and Monitor Testing
Testing
 
Code inspection
He advocates the use of code inspection by a second party, he said the use of computers would be too expensive for this task. But with the computing power we have today. Software like Findbugs or JTest can help in the first line of bad code detection and Pair Programming helps in that aspect too.

 Code Coverage
Code coverage is also advocated in fact he advocates testing every logical path if possible and if he is the customer he would not accept delivery until this step is done. Although he acknowledges the difficulty in accomplishing this step for large programs.

 Step 5: Involve the customer
He knows that software design and specifications is open to a wide interpretation even after agreements, customer involvement in his words should be "formal, in-depth and continuing". 

 Let's not remember Winston W Royce as "Father of the waterfall" but rather one who saw how software development could be done better 40 years ago.  

Relational selects in YII

How do you do relational selects in YII? The kind where you have a many to many relationship on a table. The YII page on relational queries didn't have many good examples on how to do it, so I tried it out and found it to be actually quite easy, its quite like how you would do for single table query.

The above figure was taken from the YII page on relational queries (I was too lazy to do up my own example).

We will just concentrate on the tbl_post , tbl_post_category and the tbl_category tables.

I will assume you know how to overwrite the relations function in CActiveRecord to setup the relations correctly.

So the scenario is, I would like to retrieve posts that are in a certain category. 

The idea is to use the with() method that is part of CActiveRecord, we would like to retrieve the categories together with the Posts.

So the query would look like this:

$posts=Post::model()->with(array('categories'=>array('condition'=>'name="FunStuff"')))->findAll();

In the with portion you specify the relations that you want to load in this case categories, and with it you can specify the different relation query options like select, condition and so on. In this case we use the "condition" option to specify what from categories that we want to retrieve. For this we want to retrieve all the Post with categories name = "FunStuff".

Simple rite!

LOTW - 11 Oct 2010

Design Principles Behind Smalltalk - One of the most influential language although it didn't become popular. Java, Ruby all draw inspiration from it. If you are a programmer you owe it to yourself to go try Smalltalk. Squeak and Pharos are probably the more famous of the free implementation around.

Google's answers to Oracle's allegations

Doing quite a bit of PHP programming these days so there are the few links that might be useful
Using Emacs to edit php files
Geben + XDebug to debug php code on emacs -  Haven tried this, so far with PHP, doing print statements still works for me.

AptOnCD Backs up all the packages that you have installed, making it easy to restore your packages instead of downloading them all again.

LOTW - 4 Oct 2010

A really simple introduction to Map Reduce - Great intro to what Mappers, Groupers and Reducers are all about. I was scratching my head about Map Reduce before this came about.

How big are Facebook's data centers? This FAQ will tell you all that you need to know including servers, software, location and expansion plans.

Google URL shortener goo.gl opens for business

Heard of the Arduino, unknown to me there is actually a Netdunio yeap, and it runs on .Net. Scott Hanselman talks about it.

Quote: Paul Graham on Beautiful software

"Great software requires a fanatical devotion to beauty.

If you look inside good software, you will find parts no one is suppose to see are beautiful too."


Paul Graham from Hackers and Painters 

 

LOTW - 20 Sept 2010

I missed last weeks LOTW due to lots of work and basically I was lazy haha, so this week gonna do a bumper issue. Enjoy


97 things every programmer should know - I guess most would already know about this, every programmer should go read what's in there, there are so many pearls of wisdom, now we have a video by Kevlin Henney at the keynote at jazoon talking about. Slides can be found at the video link.


Epic failures: 11 infamous software bugs


Google is going to release Wave in a Box, an installable version of Google Wave, since they have decided to discontinue Google Wave. The version is not going to have all the features of the original wave, but they are going to support federated across other Google wave boxes.


Lisp cabinet - For those who wants to tinkle with Common Lisp "Lisp Cabinet is a set of configuration files, tools and utilites bundled with automated installer to deliver full-fledged Emacs based Lisp development environment for Microsoft Windows"


Check out all the cool videos from NDC 2010 in this torrent
interested to hear Uncle Bob and Mike Cohn speak, Uncle Bob's slides from his Clean Code I Args are here.

Lamda the Ultimate has two posts that focus on Software Transactional Memory, one post quotes from a study that STM actually helps reduce errors as compared to the traditional locking model, while the other post talks about the support for STM in .NET and the decisions they have to make. I haven't really tried STM yet in any language, but basing modifications around a database like transactional model can probably make understanding and programming easier.

Designing Interfaces guidelines.

We all write interfaces when programming, these are the few rules that I found useful to keep in mind when designing and implementing interfaces.

1. Do not write interfaces for Model classes
Model like classes are classes like your DB model, or any other kind of class whoose primary function is to hold values. I usually do not write interfaces for these kind of classes for their main purpose is to hold values, and so let say you have Person class with getters and setters for Name and Address. And you have a interface call IPerson so you have getters and setters in the interface for IPerson. If you add a Telephone to Person, you need to add both the getters and setters to the IPerson. And it just goes on and on.

So far I haven really found a nice use case for implementing interfaces for simple value/model kind of classes.

2. Reduce your external dependecies
The reason you are writing interfaces is so that you can reduce the dependcies and change implementations if necessary. Consider the following

import org.internal.*
import org.external.NameValuePair

public interface HttpPost{
public void setPostParameters(NameValuePair[] pairs);
....
}

You are writing an interface to HttpPost and you have a method to add the post parameters. Your NameValuePair class actually comes from an external library. Now you have a dependency to that external library and you will need to package that every time you are using this interface.

I will try to keep this to a minimium, the reason why I write interfaces is to keep dependencies simple not to introduce new ones. So i might introduce a new class to break that dependency if needed or change the method signature to something like addPostParameter(String,String);

3. Keep your interfaces small and cohesive
Try not to have interfaces that does sort of everything, keep your interfaces small and cohesive (that means the methods in the interfaces are related to each other).

Bob martin in his article on the Interface Segregation Principle talks about using the Adapter pattern and multiple inheritance to join multiple interfaces together.

I like the line from Uncle Bob, "Clients should not be forced to depend on interfaces they do not use". Keep that as a rule when designing and implementing interfaces.

Martin Fowler Role Interface article also talks about this.

Interfaces are great for breaking up dependency and when use together with the Dependency Inversion principle you will have a easier time coping with changes to your software.

These are the book I recommend to improve your code.
Effective Java

Clean Code: A Handbook of Agile Software Craftsmanship


Agile Software Development, Principles, Patterns, and Practices

LOTW - 6 Sept 2010

Yahoo releases the Yahoo YUI Testing Interface it allows running to units tests in different browsers simultaneously.

The first step is to start from 37 signals - If you want to start something, just start, dont worry about it. As Yoda says "Do there is no try". Good advice for me ...

James Gosling creates a "Free Java from Oracle" t-shirt. Frankly I wonder what if Java was free from the beginning, would it reach the popularity that it has today? Look at those with ISO like organizations behind it like C++ or Ecmascript, they tend to have abit too much political maneuvering and arguing and it take a long time for them to agree on standards. For the one person system that Python, Ruby have, the seem to lack the mass acceptance that Java commands (because SUN was backing it up those days), yeap they are popular but are they really being used a lot in the Enterprise space?

Scott Hanselman talks about 2 tools he use for a more readable web, Instapaper and Readablility.

Google Code University  - Offers courses on C++, Python, Android, Web security, lots of good stuff comes with slides, assignments and videos.

Let's face it programmers are probably not the best designers in the world (me included), hope the following 2 links can help out.

20 useful free pdf books for designers and bloggers

10 resources for design challenged programmers

What's your motivation for programming?

Studies since the 1970s that show that Intrinsic motivation (from within) rather than Extrinsic motivation (external factors like money) is the main motivation for people doing good work. In Drive: The Surprising Truth About What Motivates Us, Daniel Pink talks about the various research and studies and even gave some suggestions on how to improve your workplace, if you are a lead or manager go read the book. If you don't have time to read the book, RSA has a nice video (about 10 mins) that sums up book quite nicely.



So what is your motivation for doing your job as a programmer? What are some of the motivations that you have for continuing to do programming as work? Here are some that I came up with, feel free to add more in the comments section.

1. Joy of knowing that there is always something new to learn

I am always amaze that there are still so many things that I can learn in this field, and that's a motivation for me, because I will always have some new framework, language, technology to play with.

2. Joy of finding a solution to a problem.

Solving a problem be it in programming, performance or a process issue is always satisfying

3. Joy of seeing a user liking your solution.

For most of us, we program for real users and its a good feeling when a user tells you that you did good work.

4. Joy of doing work with people that you like or like minded.

 I am glad to be able to work with a bunch of guys who I like and feel comfortable with, the pantry talks and the lunches help make for a more interesting day.

Exploring Google Analytics Api - 1 Client Login

Blogger dosen't have a "popular posts" widget, and since I am using Google Analytics and there is a popluar posts metric, i decided to roll my own and host it using App Engine. I am roughly following the examples on the Google Analytics Api site but adapting it for AppEngine use.

There are 3 methods to authenticate to Google Analytics and retrieving data.
1. Client Login
2. AuthSub
3. OAuth

The explanation for all these 3 types are provided here with the various scenarios on what to use.

If you are planning to serve out data from a server you should use OAuth, its the most secure and there is no passing of passwords anywhere, however since I am still playing around with the API and I don't want to mess with the complexities of OAuth authentication. I will be using ClientLogin for now.

Google does provide client libraries in Java for accessing the Google Data API (which Analytics is a part of), however they don't play well with App Engine because they use some restricted classes. So we will have to go with the protocol version, i.e using POST, GET calls to get the data.

You will need to POST the following data to https://www.google.com/accounts/ClientLogin with the following parameters.The table below from Google shows the parameters.

accountType	Type of account to request authorization for. The default is GOOGLE, which is currently the only option supported by Google Analytics.
Email	The user's email address. It must include the domain (e.g. joe@gmail.com).
Passwd	The user's password.
service	The Analytics service name is analytics. (For other service names, see the service name list.)
source	A string identifying your client application in the form companyName-applicationName-versionID.

Table 1: Post Parameters (Credits Google)

In App Engine, to do POST/GET request you will have to go quite low level, you will need to use the URL and HttpURLConnection classes. Below is a sample of using the classes to do a POST to the client URL.

URL loginUrl=new URL("https://www.google.com/accounts/ClientLogin");
   
StringBuffer message=new StringBuffer();
message.append("accountType=GOOGLE&");
message.append("Email="+URLEncoder.encode(username)+"&");
message.append("Passwd="+URLEncoder.encode(password)+"&");
message.append("service=analytics&");
message.append("source=stephenonsoftware-sitedata-1");
   
HttpURLConnection connection=(HttpURLConnection) loginUrl.openConnection();
connection.setDoOutput(true);
connection.setRequestMethod("POST");
   
OutputStreamWriter writer=new OutputStreamWriter(connection.getOutputStream());
writer.write(message.toString());
   
writer.close();
   
if(connection.getResponseCode()==HttpURLConnection.HTTP_OK){
BufferedReader reader=new BufferedReader(new InputStreamReader(connection.getInputStream()));
    
String line;
StringBuffer result=new StringBuffer();
while((line=reader.readLine())!=null){
 result.append(line);
}
reader.close();
return getAuthString(result.toString());
}else{
return "";
}

You will get back a long string containg the SID,LSID and Auth parts, you will just need the Auth part so just split the string at 'Auth=' and you can get the authentication token, you will need this token to request for the data from Analytics.

Now you have successfully login using ClientLogin, next I will post about how to get your account details from analytics.

Links:
Using java.net 

LOTW - 30 Aug

“My husband is a programmer; I have no idea what that means.” - Nice blog post about a non programmer and a programmer geek getting together and learning about each other. I agree with Renae, the important thing is to take an interest in what your other half does during work and his hobbies.

Patterns and Practices for Improving Personal Productivity, Time Management, and Effectiveness

Tim Daly writes about writing big Lisp applications.- Tim Daly responds to Steve Yegge's Lisp is not an acceptable Lisp. I let you make your own decision about whether Lisp is good. For me as a developer/programmer wanting to have more "breath" in your knowledge (I guess most of us are brought up in the OO kind of world), you owe your self to go try one of the functional languages, for me I am going to try Common Lisp and Clojure.

Pascal Costanza's Highly Opinionated Guide to Lisp - Pascal talks about why use Lisp and some of the common pitfalls and where to find more information on your discovery of Lisp. I was always a bit confused by the backquote operator ` in Lisp, his explanation is the easiest I have found. Gotta spend more time reading Practical Common Lisp !

LOTW - 23 Aug 2010

Oracle Sues Google - This must be the big thing this past week.
CNet's Report has a copy of the letter from Oracle's lawyers.
Top 5 grievances with Oracle -  A list of complaints about oracle and open source.
James Gosling foretold
Google saying that Oracle's suit is baseless.


Writing an operating system -  Something that I always wanted to try. Lets put that in the todo list !


F# in Visual Studio - Functional programming is getting lots of press these days, for the JVM; Clojure and Scala and for .Net F#, the video shows how F# can be use most effectively and introduces F# features like parallelism, succinct expressive syntax, rich access to .NET libraries.


You can get the beta Windows 7 Phone developer tools already. Singapore is one of the launch countries so we will have our own application marketplace here.


Scott Hanselman in his 56th Weekly source code talks about the Visual Studio 2010 training kit and some of the cool stuff in there. I was particularly interested in the Parallel Framework

Oracle sues Google over Android - Possible outcomes.

Oracle finally sues Google over Java. CNET has a copy of the complaint letter from Oracle. Other than a list of patents that was infringed nothing much was said about how it was infringed. I always thought that with the coming of Android, Java finally has a great platform in the mobile arena, I know people will agree with me that Java ME sucks most of the time.

I agree with Miguel that Java as a product has never been profitable for Sun, when Sun was around the big names in the J2EE app server market was Oracle, Bea and probably IBM that made money from selling middleware products. So i guess since Oracle paid quite abit for Sun and so they need to milk it for all it's worth.

How will all this pan out? Below are my 4 guesses.
1. Google pays Oracle
Straight forward way to solve the problem, but that will mean anyone else who has the same idea as Google needs to cough up money big time and that affect's Java, although people might not be moving away quickly but stagnation may occur and that's bad for any language.

2. Google goes to court and wins ! And all is well in the Android world.
Might be a long and drawn out battle, but sets the scene for even more innovation in the Java eco system. The thing is what will Oracle do about Java if it loses this battle.

3. Google goes to court and lose, Google pays Oracle
Same as 1

4. Google moves Android to alternative languages (Python, .Net).
That would be interesting, considering that Goggle is quite the advocate for Java and Python (look at App Engine). Would love to see Android supporting Python in the future, quite like how Symbian has some support for Python (although I think Symbian as a platform is dead also), the next closest alternative would be a move to C#, since they are quite close in terms of syntax and perhaps popularity, and it doesn't cost much for a Java programmer to move to C#.

LOTW - 9 Aug 2010

It's Singapore's National Day today! Happy 45th birthday Singapore!

How to win at Rock Paper Scissors -Not really relevant but it helps you beat your younger nephews and nieces at RPS.

How Yahoo does MySql HA - A blog post by Jay Jessen a MySql Guru at Yahoo.

Seven Open Solaris Home server tips  -  Tips for building an OpenSolaris homeserver with links to other peoples builds. Someone even used an EEEPC as a homeserver. Anyway I thought ZFS was cool stuff, just hope that Oracle can continue with OpenSolaris and start putting out releases. Nexenta looks like a good replacement.

IBM and DBS on the Jul 5th outage - On Jul 5th, DBS atms went down for for a period of almost 7 hours disrupting services, this is the report that attempts to explain why the outage occurs.

Learning VIM

I have been an Emacs user most of my life, every since my Polytechnic days when I was introduced to LISP (or the Language Inspired by So many Parenthesis, Yeah i came up with that!).

I decided that finally I was going to learn VIM properly, why VIM since I know emacs? Most of  my free time now is spent on a Netbook specifically the ASUS 1000H, small light netbook with an Atom processor and 2GB of ram. So i wanted a light editor. Anyway I am mostly doing PHP and Python stuff, so an Editor is probably good enough no need for some really fancy IDE.

I am starting a Cookbook for VIM at Google Docs to write down all the VIM commands that I use on a daily basis, perhaps it can serve as a reference for those who are learning VIM but don't know what commands might be useful.

Top books that I recommend for programmers

In selecting the below books, I try not to choose books that are very language specific (no python in 24 hours kind of book) but rather books that I find are general enough to be of use to most programmers.

Head First Design Patterns

The Head First series of books takes a fun approach to tackling topics, when I first got back in the programming business this was one of the first books i bought to brush up on Design Patterns instead of the dryer Gang of Four book. You get nice diagrams, background stories and illustrations to make a dry topic fun.

Clean Code: a handbook of agile craftsmanship

Great book by "Uncle Bob" for all software developers, tells you what makes code clean and readable, gives lots of heuristics and guides. Just go read it, you will become better.

Effective Java

Not just for the Java Language, contains lots of tips in general for OO languages, but in general tells you how to write better Java code. Joshua Bloch lead the design and implementation of Java Collections Framework.

Ship It!: A practical guide to successful software practices

From the pragmatic programmers, all developers know how to code (I hope :) ) but what is important is to actually ship the damm thing. This book gives a nice overview of what are the techniques, infrastructure and process that you will need in order to Ship It!. Although the book doesn't really claim to be Agile but most of the techniques they use all come from the Agile philosophy like continuous integration, small iterations, daily meetings.

The Pragmatic Programmer: From Journeyman to Master 


Pragmatic advice for programmers covers everything from using editors, version control debugging to requirements.  I guess if there is just one book that covers lots of ground it would be this. Everyone should read this, since its not dry and at time humorous.

 Coders at Work


I don't think that there is another book quite like this. The book contains 15 people who are considered great programmers, the questions for each person are quite standard and repetitive but that's just the beauty of it. You will see how people mostly agree or disagree about a certain topic. Read it so that you will learn a bit about the history of the programing craft since schools don't really teach the history of programming.

What other good books do you recommend?