Excerption: Agile Methodologies ask…

If design is good, why not make it everyone's job? (Refactoring)

If simplicity is good, why not use the simplest design that supports the currently desired functionality? (Simple design)

If architecture is good, why not have everyone work at defining and refining the architecture continuously? (Metaphor)

If short iterations are good, why not make iterations really short (hours and days) instead of weeks and months? (Short iterations & planning games)

If requirements, design, and code reviews are good, why not do it all the time? (Pair programming)

If testing is good, why not do it all the time… even customers? (Continuous unit and functional testing)

If integration testing is good, why not do it several times a day? (Continuous integration)

Test Driven 1

Test driven sounds like a good idea, but it is very hard to get used to it. When we get an idea, we could not help to realize it. We often forget to find out a way to prove what we're going to do is right. Even Nate sometimes forgets. So he often asks us don't hesitate to kick him when he forgets to write a test before writing any production code. It's not good to see a test always passes, Nate always says, that means what we do makes no difference. Every time we have to see a test fails before writing any code. So that it is our code that makes that test pass. That is test driven: writing code against a failing test.

Ruby on Rails provides so many facilities and tools to practice test driven development. There're different types of tests: unit test, functional test, acceptance test. There is a strong belief amongst Agile practitioners that every thing could be tested. So they develop lots of tools to do this. From user interface to database, there are tools to test them.

Incompatible Pair 3

In his middle of thinking, one may not like to be disturbed by questions and suggestions, and it does no good to the overall productivity as well. Nate suggested that to be a good partner, one should learn to take notes. It would be better if one writes down what he wants to talk about while the other is in the middle of thinking, as long as what he is going to talk is not so urgent that otherwise may waste a lot of effort. When waiting for checking in and out, waiting for tests to be passed, it's a good time to discuss what their doing

Incompatible Pair 2

It is still a question whether two people working together would be more productive than working separately. Currently in the company, the more sophisticated one in a pair should spend a lot of time to explain to his partner what's going on; otherwise the partner tends to be left behind or even snores away, because when he couldn't catch up, he couldn't find anything to do there. It seems that if these two people work separately, at least they can keep working all the time, which might be more productive by two. However, Agile practitioners like Nate have a strong believe that pairing is more productive than each one doing by their own. Let see how it works.

Incompatible Pair 1

Extreme programming uses pair programming. There seems to be incompatible pairs. From this Wednesday, I have been pairing with Jordan. Jordan is a passionate guy, who has been notorious in the company for working so fast, keeping the key board all the time, and seldom explaining to the partner what's going on. Now it is my turn to suffer. Of course, he is much more experienced on Ruby on Rails than me. When I paired with him, I always tried to ask what I didn't know. But he always made me felt that me questions did not deserve answers. Meanwhile, although I was not familiar with Rails, I surely knew something about computer science; so, on high level, I couldn't help expressing my ideas on how to do it or making suggestions when I found it a bad design. But he always insisted that my ideas were ridiculous, and disdained to explain why. Sometime he even made me feel that Agile Planner was THEIR masterpiece, to which I could only show my respect; he was always afraid that my naive ideas would cause THEM to change THEIR system fundamentally. He is such a poor listener that he often interrupted me before I finishing speaking out my ideas.

The problem is what I have to envisage. The problem of incompatible pair also deserve to do so study on it.

Overflow

Since we experienced a excessively slow progress last iteration, a lot of tasks are delayed. Nate called them overflows. This iteration we don't even have a planning meeting. We pick a few tasks from overflows, and add to it over time. After a depressing yesterday, Rock and I made a series of progresses today; we completed the whole story of sending emails to students when creating a composition assignment. The only thing that we had left out is a weird result from the postgres database. When we fetched email addresses from the database, for the sack of test, we sorted in query using "order by email_address". There were two email addresses in the result set: testor@163.com and testor2@163.com. In ascending order, we expected it to by in this order:

testor2@163.com, testor@163.com

And it came out as we expected in our Windows machine, so that all test passed. However, on Nate's Windows machine, it was in reverse order, which caused three tests failed. The encoding in the databases of both machines was the same: SQL_ASCII. And the two OS were both English version. We are still not able to figure it out. Anyway, Rock and I seemed more lucky than Nate today:-)

Since the timing in Agile Planner didn't work at all, from this week, we started to use Google Spreadsheets to keep track of every employee's work

We spent 10 times as we might have estimated!

Today Rock and I continued the work that should have been done on last Friday. There is a task on CIY requiring that when a teacher creates an assignment, the system should send email to every student of that course. Nate had created a skeleton of sending email for us. We were expected to spend no more 2 pairing hours on this. However, it did not seem so easy when we got down to work.

On last Friday afternoon we spent two hours on researching this part of the system, and how to send a email using Ruby on Rails, only to find that there exited some problem with database fields. What we could get is an assignment_id, with which we had to look into the database and find the email address of the students who chose that course. Unfortunately, the database was not set up as was design on ERD. The assignments table stored a field teacher_id which should have been course_id, while the students table stored a field class_id which should be course_id. Then we realized that we should use migration to modify the database before we can start to work on the mailing job.

So we started today by creating the migration to modify the database. However, when we update the local project from SVN, there were failed tests. We tried very hard to fix that failure until we found it out of our capabilities and responsibilities half an hour later. Then we turned Nate, finally finding that it was a test that had been committed before passed. After Nate fixed that failure, we came back to our machine and update again, ran all tests, it had been 9:30! We thus wasted more than an hour today just because a failed test not caused by us.

Modifying the schema of a database is a radical change; it may shake the foundation of the system. Rock and I had experienced such a nightmare on last Thursday evening when we were asked to change a field ideal_hour to ideal_day and divide the original data by 8. In that evening, we started to work in 7:30. It seemed simple initially. After spent some time checking out and passing all test, we start to change the database. Of course a lot of tests could not pass. Then we began to spend frustrating hours in fixing codes in order to pass those tests, until it was nearly 12 that night when we realized it impossible to fix those failure in a couple of hours. So we decided to roll back what we had done that evening. When we continued to work for more than half an hour, we thought that we had rolled back. Since run all the tests again may take a long time, Rock said I could leave for bed first, he could handle the rest. Not do I know the next morning that he worked until 2:30 a.m. just to make all test pass again!

So this time we were both taught that it would never be too cautious in making change to database schema, and the estimation time of that work would never too long. Before we did the migration, we decided to make a new branch in the SVN, because we did not think that we could fix in a single day all the failures of tests that were caused by changing the schema. Before we made a new branch, we had to pass all the tests and commit what we had done to the trunk. Then we could made a branch from the trunk. Then we could work on that branch.

We proceeded so smoothly this time. Rock even doubted whether there were still incorrect logics that could not figure out by passing all the tests. It is possible, but it is a test coverage issue, we were not able to do anything to it then. So we could not but trust the tests. Then it came to merge the branch to the trunk. To did this, first we had to commit what we had done to the branch, then we checked out from the trunk to another project, say admin_trunk. And then we merged the branch to admin_trunk, after resolving all the conflicts and fixing all failures of tests, we finally committed it to the trunk. It had been 3:30 then. If we started from over eight this morning, apart from an hour of meeting this morning, we had spent nearly 5 hours on it. Imaging we just modify two database fields, if we had been asked to estimate how long it would take, we might say at most 30 minutes pairing. It turned out to be 10 times we might estimate. In retrospect of how we spent such a long time, writing that two migration files only took us less than 30 minutes; we spent the rest of the time fixing the unpassed tests that were not written by us, waiting for SVN check and out, waiting when the tests were running. How to reduce such overhead time, or how to estimate such time during planning meeting, it is a question.

The second postmortem

We have encountered so many problems both technical and managerial this two-week iteration. Nate said that our velocity is 0.05 based on the figures in the Agile Planner, which means that our development speed is only 1/20 of what is expected. Though the timing of Agile Planner is untrustworthy, Nate said. It is so often that we forget to start and stop the session. Nate felt not quite good this morning when he saw that AP showed Li Pu had been continuously working on a task for more 16 hours!

We have paid so much time on configuring, nearly half of the work time. Configuring Linux, configuring Windows, configuring the company's machines, configuring our own machines. Nearly every time when we check out to start working, there are tests which should have passed in others' machine but fail in our machine. Finding what the problem is, trying to fix that problem by configuring the environment, often takes the whole four hours of work time, even the whole day.

If we can reduce the configuring time and put that time on productive work, we can significantly speed up, I proposed in the postmortem of the iteration today. As normal, we proposed so many problems that frustrated us so badly in the postmortem, and voted for the importance of each item. Because we have so many problems these few days, in the two-hour postmortem we did not have time to come up with solutions to these problems. Li Pu suggested to do it on the stand up meeting or the planning meeting next Monday. This reflected a common practice in the company, we call it time-boxing. When we try doing something, we set a time limit, called time-boxed. If we can not make any progress within the time limit, we stop it and find some way to get around it. If we fail in circumventing it, or if we find it very important to do it after reflection, then we rearrange a time to do it, and again, often time-boxed.

Subversion

In the company we are using a source control server called Subversion or SVN for short. Every time we want to start to work, we have to update the local project from the SVN server. And every time when we have made some progress and passed all tests, we update again to ensure there's no conflict by running all tests again, then we commit it to the SVN server. All tests must be passed, all conflicts must be resolved before we could commit what we have done to the server.

There's a distinction between trunk and branch on the server. As has just said, all tests must be passed before committing, or the server will reject it, which referred to committing to the trunk. But when we want to make a radical change, like change the directory structures of the project, as what Philip has been doing these days, we can't pass all the tests in a single day even a couple of days, and we want to commit what we have done to the server so that our work may not be lost, we create a branch, and commit it to the branch. When we have done the change and pass all the tests, we can merge it to the trunk, of course, all conflicts should be resolved.