Hadoop Summit 2013

k, k, k… after 2 days in Amsterdam, for the first european edition of the Hadoop Summit, I finally realised that most of the obstacles I encountered working with HDFS > HBase > Hive were due to the edge use case I was putting in practice

1) very few people use Hive over HBase

2) even less people use HBase’s Thrift interface (apart of Facebook)

they are all very powerfull instruments, but here I have to quote my current project manager… “u do not pay licenses, but u need the most expensive people on the market to have it running”

more details here… if u wanna try it out (pumping a MySQLdump into Hadoop) https://pypi.python.org/pypi/SqlHBase/0.1

mani nei capelli

ricevo questa, pretendono sia una job description… non ho parole
(ma che razza di mercato del lavoro c’e’ in Italia?)

Conoscenze: ETL, ETL Informatica PowerCenter
Skill: buona autonomia di produzione nell’ambito delle direttive date
Luogo lavoro: provincia di Bergamo
Note: ottimo team di lavoro in ambiente professionalizzante; attività di notevole spessore tecnico

hbase cluster region recovery

dear HBase developers, please… fix the documentation… http://hbase.apache.org/book/apbs04.html

[root@yallayalla ~]# sudo -u hbase hbase org.apache.hadoop.hbase.util.OfflineMetaRepair
Exception in thread “main” java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/util/OfflineMetaRepair
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.util.OfflineMetaRepair
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Could not find the main class: org.apache.hadoop.hbase.util.OfflineMetaRepair. Program will exit.

instead…

13/01/31 15:27:52 INFO wal.HLog: main.logSyncer exiting
13/01/31 15:27:53 INFO util.HBaseFsck: Success! .META. table rebuilt.
13/01/31 15:27:53 INFO util.HBaseFsck: Old -ROOT- and .META. are moved into hdfs://yallayalla:8020/hbase/.hbck/hbase-1359646067507
[root@yallayalla ~]# sudo -u hbase hbase org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair

Maildir server on OsX

hi nerds! anyone fancy of having his/her own portable Maildir backup? I’ve created a homebrew package for courier-imap (and courier-authlib). Would u mind testing it? I want your feedback! 🙂 https://github.com/zeph/homebrew

UPDATE: https://github.com/mxcl/homebrew/pull/16354

SECURITY CONCERN! :: edit the etc/imapd in order to bind only to 127.0.0.1

to start it:

sudo authdaemond start
sudo /usr/local/Cellar/courier-imap/4.11.0/libexec/imapd.rc start

and use ur normal sys credentials (and point ur config in the email client to “INBOX”)

p.s. u might need to created the Maildir folder in ur home dir … use:

bin/maildirmake

how (not) to keep employees

despite the financial crisis people like me, engineers, keep moving between companies, some people for money, some others for new challenges, but that is not the whole picture…

people need to be properly rewarder, made confident that what you do makes sense, being told the good and the bad of the company, give them hope that they’ll grow in the company and there will be always a place for them, and ultimately that the company will be always capable of paying for the bills

no, Nokia, laying off “god knows how many people”, is not a good way to gain trust from ur people… I better move on my own instead of waiting u to lay me off… and I did it

no, “startup-pb”, laying off an engineer the same day i get on board, is not a good welcoming message for me… even worse when I discover that 6 others have been dismissed in the 5 months before with the same unfaithful excuse

no, “startup-book”, telling only the good and not the bad, doesn’t help… everybody will just think everything is good

even at Google, it is not the first time that I hear people leaving cause of bad management… there is a huge disparity between the way engineers are considered, and everyone else in the company…

so, even in bad times like these, engineers, and other technical people, just move

and u are not losing only a person, or a human resource as u call him/her, u are loosing company knowledge and an investment that will be ***ing expensive for you to substitute… just look at how much head hunters pretend, and how long it will take to train or get up to speed the new person

and ultimately… yes, a nice and welcoming office space, matters