Back to Blogging

Its been a long time since I wrote anything here. Lots of things that prevented me from writing anything, like my laziness :), workload from workplace and my own stuff and newfound love with Facebook and Twitter.

So I have decided now that I will try to put in short posts, at a more frequent rate. However, I will also write some elaborate posts when I feel like and I have the time for it. If you come to think about it, there are so many things you want to write about, but its more important to get into them rather than necessarily find the time to write about them.

This is what I am into now these days.

Technology/Programming : Hadoop, Apache Mahout, CouchDB, Tapestry, Bixo, java.util.concurrent :)

Music : Lots of trance music lately, with a bit of classical. Looking for a good blues collection. Also planning upon listening a couple of Hindustaani classical ragas more intently and learning their intricacies. Found some good ear training software too.

Fitness : Started gymming today with a new determination to get into shape as soon as possible, long Diwali break broke the rhythm and ate upon past improvements :(

Financial Clarity : Planning to get familiar with basics of financial instruments and share market and assess my financial situation with plans for future.

Whoa ! big list that was ... but yes writing it out is a good way to organize thought processes. Well, time to conclude and come back with a new post later this week !

Blogging from Manhattan through a MacBook !

Its been almost five months since I posted anything on my blog. I have been through a wide range of experiences in this period, from extremely bitter to really wonderful and pleasant. I hope to be able to put in articles more consistently now, as things seem to be settling down now.

Its been 23 days since I set my foot on the American soil. I am writing this post currently from my apartment in Manhattan, New York and my happiness is doubled by the fact that I am using a MacBook ! ... I have long awaited to understand the loyalty of Mac fans and though I am very new to the fraternity (just a week), I am beginning to see why is it so. I have been to quite a few places in New York now and I have walked a lot here, which is a big change from Bangalore. This place is a sort of jungle of skyscrapers :) ... with so many people moving both above and below the roads. New York certainly deserves to be called one of the greatest cities of the world if not the greatest, which it perhaps is at present. There is so much energy moving around in this city, a place where people are living to get something done. Manhattan is a very safe place too, you can see girls walking around in streets at midnight. There are "thelaas" like we have in Indian cities, just more sophisticated. And you can see a lot of rickshaws at Times Square and in Central Park, just more sophisticated :) designer rickshaws infact ! ... a bicycle here is something which sets you apart from the crowd, just the reverse of what we have in India.

There are so many observations which I want to put down, but it will take many posts so I will keep them for later posts.

I intend to put some interesting stuff regarding Linux memory management, compilers and some ideas I have. The thing is that I want to put stuff that's my own hack (as I would like to call it) rather than something which you can find on technology sites or Wikipedia. I can't do it today but I hope I can start with something soon.

Its been a very happy feeling to return back to my blog and I hope to do better this time :)

Driver, where you taking us ?

Its a special day for me, my birthday :) ... nothing much happened today except I was able to pray in the famous Dakshineshwar temple here in Kolkata, after a long, slow walk from outside the temple premises to inside. It was a surprise for me as I had expected it to be an easy-going affair, I would be done in about half an hour, I had thought ... it took more than 2 hours, which I am thankful for, as it could have been more.

So, here I am, 26 years of my life complete today and I am in a strange state. I feel happy, I feel sad, I feel confused, hopeful and sometimes I feel blank. It seems that this has been triggered because of my thoughtful, introspective mood that I entered yesterday night. There are so many things I have thought about since. Family, friends, studies, career ... And each of these topics makes me feel a lot of things as I have said earlier ... happy, sad, confused, hopeful, blank.

I am not however, going to discuss (most of) my thoughts here. I rather thought of writing out some of the thoughts that I have for my blog though :)

Well, this is a summary of what I think I would do with my blog this year :
1) I feel like I need to write more articles that are geek-ish ;) ... what I mean to say is that, I want to write more articles that are not of a philosophical/intellectual mode ... I would like to put more concrete articles this year.
2) I would like to put a lot of small math posts that would be relatively easier for people to grasp and fun too.
3) As for programming, I would like to pick specific language features from those I know and discuss what I know and maybe that could help some new people ... maybe also invite the wrath of experts :D ... but I am ready for that, that's the way to grow !
4) Put more material on data structures as used in Linux, material on Linux scheduler and Linux memory management.
5) Some algorithms that are beautiful and less-known.
6) Cryptography - Posts introducing readers to basics of cryptography, encryption schemes, some practical and wonderful aspects of cryptography.

Hope, I can do justice to this list this year. Happy blogging to all of you and happy birthday to me ;-)

A whining post

Missed February 2008 in my blogging career, busy with exams and other stuff. I have three to four topics on which I would love to write something as soon as possible. However, it looks like even March is going to be unsparing. So, with not much time at hand, I thought of writing a post where I would list many of my recent dislikes (it feels so good to criticize ;) ... and hence I call this a whining post :)

Well, here they are in no particular order or importance :

1) Indian cricket team has created history in Australia with their sensational 2-0 thrashing of Australia in the triangular series finals. Moreover, they won the historic test in Perth which had become a holy grail of sorts for the Australians. And, the fact that India did this was even more breathtaking. The sad part is India could have won the test series also, had it not been some 'obnoxious umpiring' in the Sydney test. What irritates me now is the attitude of Indian media and some Indian fans. There are still a lot of people who dismiss Indian victory as fluke ... c'mon now, what do you want ? Indian team came back after a major mental setback in the Sydney test to beat Australia, in nowhere else but Perth ! isn't that awesome ? Beating world champs in ODI versions 2-0 in triangular series finals emphasized the domination of Australia by India. The drama by Indian media is sickening, ridiculous, irritating, over-hyped, emotionally exploiting and what not ... the post-victory shows on leading news channels like Aaj Tak and Star News was senseless. They were blowing things out of proportion, I mean let the team relax, don't play with the sentiments of people for your ratings ... these same channels will not leave a stone unturned the next time Dhoni and gang fail ... the way they projected the clash between Harbhajan and Symonds, Hayden was shameful ... yes, Australians are bad losers but you got to be sensible with national broadcasts. The Indian media looks to me like a bunch of a**holes who flush the ethics with their shit.

2) I love Ubuntu, this is the most satisfying Linux distro I've laid my hands upon (well, definitely some people won't agree but that's ok with me, you got your reasons to stick with your distro, have fun). However, there are somethings I would like to be improved. Firstly, and I am talking about Ubuntu 7.04, why is the shutdown so ugly ? On Windows XP, the application windows get closed one by one in a clean manner, the shutdown on Ubuntu is like a mad rush to wipe out the window graphics from the screen (you get white rectangles all overlapping, some colours too and then suddenly to the black screen, the sound is broken and sometimes there is no sound !). I have done a bit of searching and come across some links that I am not sure are talking about the same problem ... I would post about this if I can get it fixed. Secondly, why isn't there a group by name or group by type option in the Nautilus window manager in GNOME (even KDE doesn't have it) ... this is a comparison to Windows XP but that feature helps to make the folder view look more organized.

3) There's this hand gesture that I see all the time when I am at some rock band show (be it in Bangalore or Kolkata) that really irritates me. A lot of young people do it, this is done basically by closing your two fingers just before the little finger along with the thumb, so that the index and little finger are pointing forward ... it's meant to show your appreciation to the band, but I intuitively feel that most of these people are bullshitting, they don't even understand the music. I am not a big guru of music but I would rather stay calm and clap when I have enjoyed some good music from the band ... it makes more sense because you are more likely to be awed and happy by a good/excellent performance rather than 'yo man'ing the performance ;) ... and I find more support for my theory because you can find these people behaving in the same fashion over a crap performance.

4) The obsession of newspapers and media with politics and celebrities is disgusting ... what do millions of people who are working hard for their meals every day care about Britney Spears' baldness, Paris Hilton's exploits or Kareena-Saif romance ? Go to hell ... I wish there would be a lightning some day that would crack on each of these media people and these narcissistic celebrities who occupy unnecessary attention.

I feel like I would spit out a lot more whining if I continue ... so I am cutting it short now and switching to work.

Where are your tools ?

A new year brings with itself a new hope, a revived enthusiasm to set up things in your life in order. That's exactly the purpose of this post. I intend to help some students who are willing to strengthen their programming skills this year.

I have observed that a number of students in colleges who like to program in C, C++ are a bit short on the knowledge or are in complete ignorance, of how to use tools that are used by professional programmers daily. These tools like make, gdb, prof/gprof, valgrind/Electric Fence, CVS/Subversion really help you a lot with programming projects and in fact programmers can't imagine doing their work without these tools. You seriously need them for large-scale projects, and even with smaller projects you will notice how helpful they are.

My plan in this post is to give such students a short intro to these tools and places where they can start learning these tools. And as you may have noticed, there is bias towards the Linux/UNIX environment here. That's because it's really a much more powerful environment for programming than Windows. You need tos get good in Linux/UNIX environment if you really want to learn more and better about C, C++ programming.

Note: This book, The Linux Programmers Toolbox , by John Fusco, is a must-have for all those who are looking to do some serious programming in the Linux environment, and covers all the tools I mentioned. It's very well written also.

So let's start with 'make'. When you are writing an application, you will have many source files, that roughly depict how you have broken your app into modules. Now, as will happen almost always, you will need to make changes to some module of your app to get things working, or maybe adding a new feature. Once you have made the changes you will need to recompile. This means you will recompile all your source files (in case you don't know make already) which is a waste of time and CPU resource because you changes only one or a small number of modules in your app. And, so ideally you don't need to recompile the ones you haven't changed. This is where make is so helpful and the importance increases when the size of your application's code increases. The book, C in a Nutshell, by Prinz and Crawford, is also a good resource for learning about gcc, make and gdb. You should try to use GNU make as it is the one used almost in all open-source projects. The page is, GNU make, and has documentation and other relevant information on it. Even if the documentation looks a bit heavy, make sure you do read it every once in a while to know more about the capabilities and intricacies of make, it will help you in the long run.

Now, another very essential tool, 'gdb'. gdb is the GNU debugger. For those of you who still do the debugging by using print statements, it's important you know how to use a debugger because a debugger can save you a lot of time and labour. That's because debuggers will display the values variables are having at any point in the code and so you don't have to guess which variable values to check through print statements, removing and re-writing print statements in your code again and again. Also, you can put breakpoints anywhere and let a particular section of the program run to get an idea of what's going on. Once again, the book C in a Nutshell, is a good place to learn gdb and once again the GNU page manual, gdb, is the authoritative source.

Profilers become important when you want to improve the efficiency of your code. You need to identify the hotspots in your code, that is, the portions of your code which take up most of the execution time. Then you can think about why it's taking that much time, and try to substitute better algorithms. prof, gprof are some common profilers available on the Linux/UNIX systems. You can learn about gprof from this great doc, GNU gprof, by Jay Fenlason and Richard Stallman. This SUN doc, is also a very good one.
Let's move on to memory debuggers. Electric Fence is a memory debugger that triggers a program crash when the memory error occurs, so a debugger like gdb can be used to inspect the code that caused the error. You can learn about eFence (as it is called) on this page by the author of the software, Bruce Perens. He also points to some tools that are better than eFence, however eFence is simpler and a good starting point. 'valgrind' is actually a suite of tools that can automatically detect many memory management and threading bugs, and profile your code in detail. valgrind emulates a CPU and so really slows down your program. You can do some tuning to get a bit improved runtime with valgrind. It's heavy-weight compared to less memory-demanding tools like Purify. valgrind consists of five tools, memcheck (a memory debugging tool), cachegrind (a profiling tool which provides information on cache hit-miss and branch prediction events), callgrind (a tool that shows the call relationships and costs), massif (a space profiling tool which provides information on parts of your code that allocate memory) and helgrind (a debugging tool for threaded programs). You can download valgrind from the official site, and also find the very detailed and good manual there. Again, the manual may look heavy but you should read it.

Well, all the above tools help you with the programming logic and builds. You also need a good tool to manage your code when you are working on a considerable sized project, since you will be revising your code and you need versions of the code so that you can fall back to a code that worked. Also, in the case where you are working in a team a tool is needed that can provide safe access to code to the programmers in team, that is, one person's edits are not interrupted by other and everyone gets latest version of code. In a nutshell, you need a verison control system, like RCS, CVS, Subversion etc. RCS is probably the oldest, CVS is the most tried and tested and Subversion is the newer one with a set of exciting new functionality not in CVS. In the beginning, stages it really won't make a difference which one you pick but eventually you should use CVS or newer alternatives like Subversion. The official Subversion book is a good place for learning Subversion. Some good places to learn CVS are the cvshome.org archive, the Utah University page.

Ok, so with that I end my intro to these tools. I hope I have sparked some interest in some of you and you will go and try using these tools. And believe me, you should learn and use them because they are really necessary when you work on bigger projects. You will save a lot of time and improve your skills. Plus you will feel more like a professional programmer and that's cool, aint it ? ;-)

Revolution Lies Within !

I have been reading many of Steve Yegge's posts lately. Most of his posts that I have read, are in essence (and this is related to programming):
1) Think about your work.
2) Improve the depth of your knowledge by learning more on the subject.
3) Improve your programming wisdom by training yourself in various tools/languages.
4) Improve your maths, learn as much maths as you can.

The first two points are generic in nature. Even the third is generic if you try to understand the meaning at its core. The fourth one would be a bit surprising for many people but it is in fact the most radical of those points. This does not mean others aren't important but this one is an unorthodox advice. I want to talk here about the last two thoughts, as everyone is familiar with the first two (even if they don't follow them ;-).

You can have a look at the posts (and I really recommend you to go and read them).
Math For Programmers
Rich Programmer Food
Software Needs Philosophers
Execution In The Kingdom Of Nouns
Ten Challenges
Tour De Babel
Being The Averagest
Practicing Programming
Math Everyday

As I said earlier, the third one is generic too because it illustrates the importance of being open-minded, unprejudiced, unbiased. Here are some quotes (I found them on an interesting blog by Reg Braithwaite and they quite illustrate the third point),

Programming languages teach you not to want what they cannot provide. - Paul Graham, ANSI Common Lisp

An “X programmer”, for any value of X, is a weak player. You have to cross-train to be a decent athlete these days. Programmers need to be fluent in multiple languages with fundamentally different “character” before they can make truly informed design decisions. - Steve Yegge, Code's Worst Enemy

What I can conclude from reading these opinions and myself having a small look at functional programming is, that you definitely need to learn many different programming languages with different paradigms. You do not want to be one of those five blind men with the elephant, isn't it ? And, I really hate it when I see people raving over Java. I am not a Java-hater, though the language has got its own problems as many other languages do. I hate the sickening attitude of those Java-philes, because they look like a bunch of zombies to me. Zombies, zombie-d by Java, fanatically convinced of Java the Panacea, who have built a wall of 'bytecodes' around them and pretend that this enclosure with what it encloses is all that there is. Most of them are newcomers and less-experienced, as those who have been deep into the rabbit-hole of Java surely know that all is not well. Now, you may question my experience as I myself don't have years of experience behind me and that's perfectly valid. However, you have got to pay attention when senior programmers (those who have been there for more than a decade) point out problems with your current favourite language. As when Alexander Stepanov, the implementer of C++ STL and a hugely respected guru of generic programming, said the following:
"I find OOP technically unsound."
"I find OOP philosophically unsound."
"I find OOP methodologically wrong."
"I have yet to see an interesting piece of code that comes from these OO people."
"I think that object-orientedness is almost as much of a hoax as artificial intelligence."

When all you know is language X, you will fire up code without thinking whether X is a good candidate to solve the problem at hand. You will think in the terms of X, you will try to fit the problem into the mold of idioms of X, into a particular paradigm. When the only tool that you have is a hammer, everything looks like a nail.

Coming to the fourth point, its really appalling to see that people just don't know or want to do maths. Whatever they learned, software jobs have washed it out of their minds. Its a great loss and they don't realize it. As an example of why should you study some maths, consider that you have a new problem at hand. Now, most perhaps you will come up with a simple, naive solution to it and use that. This will show up as a problem when considering the efficiency of your application. Had you known some maths, you might have tried optimizing or finding a better solution to the problem. You can say that you can workaround this problem by doing a search on algorithms for such a problem. However, that's not very easy unless you follow points 1 and 2 listed above. Also, in the worst case it may be a new kind of situation and then you are left helpless. If you knew maths, you could try to formulate the problem in some mathematical models you knew, and try developing a better solution. Most wonderful would be the situation where you could model it from scratch and develop a solution. As a more practical example, suppose you had some algorithm (or data structure) which work on the kind of problem you have at hand, but the problem isn't directly fitting into the domain of those algorithms or data structures. You could tweak the data structure or algorithm to fit into your requirements, you could analyze the costs involved ... all possible if you knew maths. Computer Science at a fundamental level is maths.

People with engineering degrees from reputed institutes should be more technically-oriented rather than running for those management courses or jobs (its entirely my opinion). It just doesn't feel right to me. Our country needs a lot of technically skilled people to carry it into the league of advanced nations. The fire of real progress is going to be ignited through science and technology, not through management. Management makes sense only when you have some strong foundation, real growth and progress. So many managers and not enough technically-skilled people is a sure recipe for continuing to be the copycats that we are today. Little wonder why we don't have world-class software products, cutting-edge technology.

Ok, let me wind up. Forget Java or C++ or Ruby, the point is that you need to have an introspection session regularly, where you stop working in the routine manner, and you try to think. Try to think whether you are on the right track, are you working with an open mind, is there something you are missing in your toolset, is your language really giving you what you want, do you need to know a new area of maths ... questions like these. I feel that this kind of checkup and acting upon the conclusions from it honestly, can make us wiser and help us grow.

On Happyness and Generators of Groups

It usually happens to me. There are days when all I can think about is completing the work at hand or about my course of action for coming days. So I just keep on reading stuff, finishing assignments, preparing for exams, working on some of my ideas, dreaming and day-dreaming about rosy career-prospects ;-) ... and then somewhere in the middle of it, I realize as if I am trying to run as fast as I can without enjoying what the 'present' has to offer to me. It doesn't look very good and then I try to take a sort of small break and as a result some funny ideas come to my mind. So, this is a recollection of one of the recent such moments I had.

I was trying to learn some functional programming few minutes before this break that I mentioned. I pondered over the reason of why I wanted to learn functional programming. I answered, "because that would make me a better programmer". Okay, but why do I want to be a better programmer ? I answered, "because I love programming and it's really cool to know that you have learned something important, you have enhanced your skill, you have a wider perspective". Fine, but then I questioned, why does it matter really that I have gained a broader vision about programming a solution to a problem. To cut the chase, it finally dawned on me that the reason was "Happyness" (yes, that spelling is borrowed from the movie). Moreover, it dawned on me that in fact, Happyness was the answer if you started to ask questions and reduce your pursuits to some fundamental reason. I am very sure on this that everyone of us can apply this method of reasoning to themselves and the answer will inevitably turn out to be Happyness. It may be studying maths, working hard on your tennis games, practicing new moves in chess, staying late at office to finish some task ;-) ... whatever.

Well, that's half the story. A remarkable characteristic of human mind is that it's working is strongly based on patterns and it learns a lot through drawing analogies. Surprisingly, a strange analogy struck me in regards to the above mentioned stream of thought. Let me now bring you to the topic of groups and generators which will help you understand my crazy analogy.

Algebra is a beautiful subject indeed. It represents an example of what the mainstream mathematics of today looks like, which is, abstract. Yes, modern mathematics is hugely abstract in nature. It may surprise some people as mostly it is thought to be involving calculations. In fact, this was the scene a few centuries back. However, over the course of time, the emphasis has moved from computing to understanding, which brings in abstractness. Don't misinterpret me in that there was no understanding involved in mathematics a few centuries back. It's just that mathematics then was mostly used as a tool to calculate answers to problems, mostly physics-related. What happened that mathematicians studied some related problems and then in the process of study, refined their results more and more to some basic essentials, which became the abstract theories of mathematics we have today. In Algebra, we study mainly algebraic systems (such as groups, rings, fields), which simply put are, a set of objects together with some operations for combining them.

Which brings us to Groups. Now, I am not going to bomb you with heavy mathematical definitions. I will try to explain to you in as much a layman-style as possible. Consider the set Z11={1, 2, 3, 4, 5, 6, 7, 8, 9, 10}. Now, we will add an operation to this set ... it's multiplication (and you thought it was going to be some Lebesgue integral over the norms of the p-adic variation of this group !). Yes, plain old multiplication but with a little twist, it's called multiplication modulo 11 (in general modulo n). This is nothing complicated, all you have to do is that whenever you multiply any two numbers in the set Z11, you take the remainder of product divided by 11 as your result. For example, 2 x 6 = 1, because you take the remainder of 12 divided by 11. Similarly, 3 x 5 = 4, 4 x 7 = 3 and so on. Guess what? We have a group at our hands and it is the pair (Z11, multiplication modulo 11). Technically, there are some other properties this pair needs to satisfy to be called a group but, I won't be going into those and you can believe me here in that this pair is indeed a group.

So, with the group at our hands, it's time to look at the element 2 in the set Z11. 2 here has a special property as does the group Z11. I am going to denote power operation by ^. Observe now. 2^0=1, 2^1=2, 2^2=4, 2^3=8, 2^4=5 (not 16, remember we divide the result of multiplication by 11), 2^5=10, 2^6=9, 2^7=7, 2^8=3, 2^9=6, and 2^10=1 again. What we have here is that a single element (2 here) generates the whole set when the operation is applied to it repeatedly. The power thing is actually 2 being multiplied with itself again and again. Thus, 2 is special and it is called the generator of the group (Z11 itself is special since it is an example of groups that have generators, many groups don't ... in fact put any prime in the place of 11, say p, and you will see that 2 is a generator of Zp).

Okay, let me get to my crazy idea that I was going to talk to you after all. Probably you might have already got it or a feeling of what it is. To kill all the hyped suspense, the idea is:

Happyness is the generator of the "group" formed over the pursuits or actions in our life.

Crappy, isn't it ? Yeah, sorry to bother you all along but let me give some reasons as to why I think so. First, you can see the strong analogy between the questioning process I mentioned and the generation of group by a generator. What I mean to say is that, each of your actions fundamentally derives from the a desire to be happy. Hence, happyness when applied to itself through an operation (that I am not very clear about) generates all your other desires, actions and pursuits. Secondly, sometimes you find that at some stage of your questioning you may have more than one reason/answer for your question. This can also be explained in the analogy if you observe that 4^2=5, 4^3=9, 4^4=3. So 4 also generates some elements but not the whole group (try it, you will know why, it's not tough). It is 2 that generates the whole group. So is it with Happyness, it is the "fundamental" generator, or in other words, the generator of the (whole) group.

Now, you may pose a valid question ... if we consider one person's desire/actions to be a group and do the same for another person, is it the case that the happyness element that they both have in their groups are the same ? I mean to say that whether when you feel happy and I feel happy (when the happyness element is expressed), is it the same feeling ? Well, I should say that analogies when carried too far or stretched too much, usually break. So, I am doubtful if this one holds. On a different note, this question is a very important one in Algebra, as it is basically the question of two groups being isomorphic (and I don't have the time to explain that here, maybe some other post :-)

"This person has gone nuts!" ... must be the thought coming to your mind, so let me say goodbye and thanks for reading.