Sunday, August 9, 2009

Programming

I've actually written code lately, although for some reason it's been all these stupid projects. First I needed to fix the kernel tty refcounting, then I got all OCD on the git SHA1 routines.

I don't quite know why I wasted that much time on something as trivial as SHA1 hashing, but it was kind of fun in a "let's use the compiler as a glorified assembler" kind of way. Some people seem to think that C is a real programming language, but they are sadly mistaken. It really is about writing almost-portable assembly language, and it turns out that getting good results from SHA1 really is mostly about trying to fight the compilers tendency to try to be clever.

So here is the current result of me trying to get gcc (well, arguably of it is mostly the C pre-processor, rather than the compiler proper ) to generate good assembly code. On my Nehalem machine (but not Netburst or Atom - poor fragile micro-architectures that they are), it actually seems to outperform the OpenSSL hand-written assembly language implementation.

And once I get rid of libcrypt from openssl, I get rid of two silly runtime loadable libraries that git no longer needs. And that in turn speeds up the test-suite by a couple of seconds.

Did I mention that I seem to have some OCD issues?

49 comments:

Adrian Nowak said...

Lots of issues, but what about Your's vacation? ;)

alvaro said...

now that the issue with the ttys is resolved, is alan returning as the maintainer?

vjt said...

@Adrian nerds never go on vacation.

And they're everyday on vacation, as well ;).

fynali iladijas said...

That's right @vjt; besides, rest is nothing but change of work! (-;

Crispin Stichart said...

If someone weren't OCD, nothing would ever get done. ;)

Unknown said...

Why not do some real good and turn that OCD on the tty layer?

Christopher Olah said...

Mr. Torvalds,

I'm very curious about something: how much CS do your children know?

It's interesting because they're around someone who's extremely well qualified to teach them. Surely, they must ask questions about your work? Does that translate into any real knowledge?

bert hubert said...

Linus,

You might want to look at http://cr.yp.to/qhasm.html which is more exactly what you discuss, "high level assembly that does not get in the way". It outputs normal assembler.

I bet you could write a sha1 that is even faster that way, and with less pain.

Cheers,
bert

Juz said...

Greetings from Cape Town.
OCD + Nerd-ism = great success.
So keep at it Mr.
Juz
http://juztick.blogspot.com

Aviv Ben-Yosef said...

Nice work, but I'd refactor and shorten all those rounds to something simpler. Seeing the permutations does help understand what is done, but anyone hacking that code should know how SHA1 works. No need to spend 100 lines repeating the obvious.

Modern Man said...

In programmers, as in airplane mechanics, a little OCD can be a good thing.

Chandra Tewary said...

Hey..you are right in either way :-).If real implies machine level or more high level english type interpreted one.C is not real :-(.

Anonymous said...

Programming, what is it??? - A lot of water has run down the "Vantaa River" since I sat in "Helsingin Yliopiston Tietojenkäsittelyopinlaitos" in Töölö in the 70s. As a humanist an a Lutheran theologian I didn't care about "Automaattinen tietojenkäsittelynoppi" back then... but the female students were intellectual.

Osman said...

Haha Linus, people say I have OCD issues too.

I try to optimize my code down to every last instruction. I often look at the compiler output to see what assembly is being generated.

I work in embedded systems though so I can get away with that kind of OCD. If I worked on PC applications I think I would go insane with the amount of inefficiency.

Chandra Tewary said...

I think when you told that C is not real prog lang then you meant it is not Turing Complaint. But, it is really hard to prove C is not Turing Equivalent. Can you please give one example?. I got to know from search that it can not implement Turing Equivalent recursive functions due to bounded sizeof(void*). Supporting C for TC by implementing Prob Of Tower Of Hanoi : http://bit.ly/5pxIC . The C code for that is http://www0.us.ioccc.org/1995/vanschnitz.c.

Thanks for your reply in advance.

Unknown said...

You have Obsessive-Compulsive Disorder???

;)

souriz said...

Hi, Linus! Are you using the Twitter?

Unknown said...

Listen to me "mr.god". Linux is fail since you got owned by Alan Cox. He owned you and you couldn't handle it. He solved a problem easy and it was too much for your pride.

Look at the design of linux.org and kernel.ugly.org! I could take a 1hour course from my friend in c++ and I WOULD RAPE YOUR LINUX system today. GET ALAN COX BACK, or you are nothing but a fat little finnish nerd sitting there and thinking you are a god.

PS. I know why you married a kickfighter, because you are a little pussy that need your wife to fight for you..

HAIL ALAN COX! IM HERE FOR YOU BRO <3 R.I.P @ true kernel hacker.

John Paul Alcala said...

OCD is of great benefit to us programmers, depending on how we use it. Applied correctly, and we can go where no one has gone before.

LOL

Marlon Ferrari said...

Hey Linus... I'd like to know from you if the correct is GNU/Linux or Linux. Thank you.

andrew ian chernin said...

Linus
My name is Andrew Chernin and I'm a reporter for a Chilean current affairs magazine. I was writing to you because I wanted to see the possibility of interviewing you, since Linux is turning 18.
Please let me know if there is any chance to work a phone conversation for next week.
Best,
Andrew Chernin
Reporter for Qué Pasa Magazine
Santiago, Chile
56-2-5507503

Steven said...

> I'd like to know from you if the correct is GNU/Linux or Linux. Thank you.

Give him a break. he wanted to call it Phreax. Which would you rather use ? It's "L-i-n-u-x"

cesar said...

Please, linus, give me your mail, please, pleaseeee!

What do you think about the new ChromeOS?

pls email me to emuletas.huineambar@gmail.com

Unknown said...

Lol, what's up with all the people trying to whine and contact him through the comments of his blog? lol give him a break!

Rob's World of Tech said...

@Alexander: WTF? Maybe Alan Cox will be missed, but the kernel will march on without him. There are lots of talented folks working on the kernel these days.

As far as OCD goes, I often wonder why things like git are written C in the first place. I can see some routines like SHA1 being accelerated with the use of C code, but realistically, there's no reason that 90% of git couldn't be written in Python, Perl or Ruby. Faster development, less bugs, no need to write stuff like SHA1 routines as they're alreay in the library.

John Bueno said...

There must be something wrong with terminologies, hasn't this been standardized yet?

Well, the OCD part is but natural... I think anyone of us get that from time to time
=P

Unknown said...

You are a GOD !
Thanks for your help !

gih said...

Wew!.. programming.. I can't take it.. My brain would come into twisted in codes.. gosh!

Unknown said...

I spent the evening tuning /sys/block/md? and the knobs in /proc/sys/{vm,net} to try to gain RAID/iSCSI performance. Turns out I managed to get a good boost and then managed to lose it all and then some.

As for OCD, I share your pain. At least you made forward progress.

Alan said...

It could be worse. You could have "CDO". "CDO" is like "OCD" except the letters are in the correct alphabetical order.

Hot Cover Girls Central said...

Just dropping by!

See yah!

Cathy Young
http://hotgirlsphotocollections.blogspot.com/

Flavio J. said...

Poque o pessoal nao faz uma instalação mais simples do slacware? é uma otima distribuição, mas ficam dificultando o acesso ao usuarios iniciantes.

The pale observer said...

OCD is a lifestyle...

Ronald Landheer-Cieslak said...

The SHA1 code actually looks really clean. I'm surprised it would out-perform libcrypt!

Shayne Wissler said...

I for one sure would like you to get OCD over Linux audio ;)

Unknown said...

Itis pretty evident that you're not just OCD about SHA1 routines ;)

Daniel said...

Hi there,
I did a C++ experimental version of this, using g++, with the 'template metaprogramming' technique.
http://pastebin.com/f35e12aea

Unfortunatelly, I can't get g++ to generate a single function, but splits it in 3. Nevertheless, it's interesting to see what code it generates. I'll compare both assemblies later.
Daniel.

Daniel said...

(of course, -O3)

Daniel said...

Despite I didn't do the performance comparison, seems that the C++ code performs really well (far better than the loop-based C code).
Interesting here is that I didn't use the C++ compiler as a fashionable assembler ;-)

zeenix said...

Some people seem to think that C is a real programming language, but they are sadly mistaken. It really is about writing almost-portable assembly language

Niin totta!

milkandoj said...

Jesus Christ Linus, if I were you I'd put down the keyboard, let go of logic, and learn how to draw or play a musical instrument or some shit.

Give yourself a break and pursue some creative endeavor that you're naturally terrible at. :P

Anonymous said...

cialis kaufen
zithromax rezeptfrei
lovegra farmacia
silagra online bestellen
cialis vendita farmacia
vigora pharmacie
antabuse prezzo farmacia
acheter pharmacie france
clomid vendita
costo farmacia
viagra vendita italia
trandate senza ricetta

CB said...

I for one think your OCD with the kernel is what has made it what it is today, excellent.

So keep up the good work!

I have read the reasons that many things, sound and device drivers being two, are not part of the kernel and have to admit that it makes sense to me.

However if you do decide to take Shayne Wissler's suggestion and fix audio, please, oh please, allow me to listen to music, while playing video and still get a VoIP phone call (Skype, Asterisk, whatever) without the system crashing. Ideally let me select which of the three playing audio streams to listen to without any stopping.

Another sign you must be doing something right, as if you needed it, is that kernel terminology seems to be creeping into the Windows vernacular. I for one will not be surprised to learn at a later date that they have been wholesale using even more open source code in their proprietary code.

A sign that if you can not beat them (Gnu/Linux) you join them, except they are still fighting that aren't they.

DAndC said...

what is your opinion about GNU hurd..

Anonymous said...

Acheter lamisil France
Comprar cialis super active plus Internet
kamagra Rezeptfrei Bestellen
Achat anafranil en ligne
kamagra Rezeptfrei Kaufen
Precio kamagra oral jelly Espana
cialis super active plus Rezeptfrei
Achat kamagra en ligne
Acheter du lasix en Pharmacie
Achat lasix en ligne
nolvadex Sin Receta

Unknown said...

C++ template meta-programming forever! ;)

Anonymous said...

за Антония Владимирова, аз се записах на мечтите фондация гости през 2005 г., и тя ми краде Бог говори защитени chrsitian министерство име, министерството на сънищата, Исус дойде в мен, като тялото ми перфектни размери и Rev. Тони Владимирова се опитва да се рушат, но призовава си преподобният, кажете на нея, показват мейла си, и нека я лъжа, никога няма да получите на Светия Дух, които твърдят, че са му Рейчъл Александър, каза, показа, hollie му shes ъ-ъ вещица, hollie, заяви, показа, Рейчъл Александър показва отсече, като му боядисана коса, каза hollie, давай напред и треперят, каза Исус, Im навой този в мрежата, този Исус typeing на компютъра, следете за това , Рейчъл, каза Исус, shes Александър вещица, каза hollie му Пол Муни, каза hollie, отрязани, като, жена си, Micki съди Муни, ха-ха-ха, казва Господ ми хареса си, rache, дявол ти, ти, казва Господ, аз знаех, каза, Рейчъл Александър, about antonia vladimirova, I signed the dreams foundation guestbook in 2005, and she steals my God spoken protected chrsitian ministry name, ministry of dreams, Jesus came inside me like a body my perfect size, and rev. toni vladimirova is trying to ruin, but calls herself reverend, tell on her, show her email, and let her lie, you will never get the Holy Ghost that you claim to have, its rachel alexander, said, showed, hollie, its shes uh witch, hollie, said, showed, rachel alexander showing cut off, with, its dyed hair, said, hollie, go ahead and tremble, said, Jesus, im puting this on the net, thats Jesus typeing on computer, watch for it, rachel, said, Jesus, shes the alexander witch, said, hollie, its paul mooney, said, hollie, cut off, like, her wife, micki sue mooney, ha ha ha, saith the Lord, I hated your, rache, you devil, you, saith the Lord, I knew, said, rachel alexander, they are the children of their reverend, its, dr. rev. jeremy taylor u.u. he steals from a christian and gives wife webby, kathy taylor, but, jeremy taylor, d.min is, at, wisdom univeristy, tricking for high fees, for loot, said, his, lovely, kathryn taylor, put it under, said, hollie, they are witchs, said, hollie,

Manas said...

Hello sir

I am working on to replace a linux kernel with the cut down version open solaris kernel in android for my project

Is this feasible ????

I am testing out other options for android kernel dev..

Anonymous said...

For the last couple of days I've been searching for Programming and finally I stumble into your blog, it has great info on what I'm looking and is going to be quite useful for my studies.
BTW is amazing how many generic viagra blogs I manage to dodge in order to get the right site and the right information...lol
Thanks for the post and have a nice day