one giant leap for kde

i am back from Berlin and settling into my cozy nook in Canada again, staring down the barrel of KDE4 development and trying to piece it all together ...

for KDE4 to matter, our development processes will need to continue to evolve. while KDE has proven itself proficient at creating technology thus far within the confines of what we have needed to accomplish, we have (broadly) three distinct challenges that KDE4 will bring with it:

The Developer's Challenge: Being Creative. there is a combination of feeling there is safety in going where others have been and doing KDE development as a sideline rather than a primary creative focus that contributes to a distinct lack of creativity in how we put our applications together. this has been, in a way, a godsend for KDE up until now as it has let us catch up with pre-existing closed source environments by pragmatically reproducing analogous features, often in better ways. but as we approach the state of the art, to go further requires creativity. we will have less and less to copy from as we launch into the brave new world of being leaders on the desktop. KDE4 needs to be more than just KDE3 updated to Qt4.

The Designer's Challenge: Creating Clarity. many of our user interfaces are a mess. they are featureful and interesting, but many times lack the quality of "clarity". at the same time, clarity has become one of the primary decision making factors for users of desktop software. as all other things have become increasingly equal, clarity is becoming a more important issue. while we have continued to expect every developer to create their own interfaces (which has led to our current state), there is now a need for interface designers to step up and produce clarity in the interfaces. concurrently, there is a need for coders to respect and support those efforts.

The Artist's Challenge: Breathtaking Beauty. the appearance of our desktop has come leaps and bounds forward. the next step will be the hardest: to create something so instantly welcoming that it inspires the user. Apple accomplished this when they first release MacOS X, for instance. and it wasn't just great icons and a funky theme, though that certainly was part of it. it was also widget and window transitions, visual concepts in the finder and more...

each person in the KDE project is faced with at least some of each of these three challenges. it will take a unified push between coders, artists and designers to fully realize the potential in KDE4.

and this is what has been on my mind over the last few days: where will i draw my creativity from and focus it on? what areas of our interface could i help make clearer? how can i support and work with the artists in our community to help them achieve the goal of breathtaking beauty?

big questions indeed ...

... and then i woke up somewhere

fortunately there was both a laptop and a network available. what would i have done had i woken up in a disconnected state?

these keyboards are driving me nuts, though. ß

Best Wishes, Aaron and Scott

Studio To Go! ... another KDE success story

i just read about Fervent Software's cool Studio To Go! product. it's a LiveCD for musicians: pop the CD into the drive and boot the machine and next thing you know you have a KDE desktop with tons of quality music studio software, including Rose Garden, Lilypond, Jack, K3B and a ton of audio utilities. many of these applications are KDE apps, and the whole experience is wrapped in a no-nonsense KDE desktop (v 3.3.1).

they are offering their product for the asking price of £49.99 and recently got a 5 fish rating from LinuxUser Magazine.

i'm always happy when i see another professional quality product using KDE, both the desktop and the applications, as their foundation. best of luck to Fervent! and congrats to all the Open Source hackers who have created the software that has enabled products like Studio To Go! to come together.

the ISVs approach

Adobe readies version 7 of Acrobat Reader for Linux/UNIX. finally, an update after languishing 2 versions behind! replacing software that looks and works like its years out of date!

Nero releases a Linux version of their oh-so-popular software for Linux.

the ISV's are coming! the ISV's are coming! hooray!

so i download these bits of software, hoping for the best. after playing with them i feel let down. why?

KPDF looks better than AcroRead7 and even competes for rendering speed on most of my PDFs. AcroRead7 has more rendering features (e.g. forms) and better rendering performance, but otherwise it's a shameful piece of software. if you ever want to see what not to do in a GUI, download AcroRead7 for Linux and explore it's "interface". glaring, horrific mistakes are to be found everywhere.

then there's NeroLinux, which K3B utterly takes to school. K3B is generally faster, has more features and looks better. bah.

now we have a really odd situation here. the ISVs are starting to lumber into Linux Land with their closed source software, something we've been hoping would start happening for a few years now. but their offerings are currently subpar, and i doubt they'll see huge adoption rates because of this. especially as the Free Software alternatives keep getting better and better. they are entering a market as second-movers (or worse) and seem to be doing so somewhat blindly.

this presents us (the Open Source desktop community) with a conundrum. if the ISVs don't see an ROI in the form of user base adoption, why should they bother? if all they get is negative press, why not slink back to WinLand and their MacDaddy where the glossy fan-mags will be happy to give them 4 and 5 star reviews?

obviously, these new comers aren't familiar with how to develop for our platform. and they are making some really questionable choices right out of the gate. this is probably not because their developers suck, but because they don't have the experience necessary to do the work well.

somehow a bridge needs to built between the ISVs who are arriving at our tables and the community of developers who actually know how to do this stuff properly. we need to have wage-slave appropriate documentation (which is radically different than enthusiast-appropriate documentation) and better support for ISVs who need to work in semi-secrecy prior to product launch. we don't have either of these things. for shame!

in a perfect world, the ISVs would pitch us some manpower (or funds to finance the manpower) to create these things. in a perfect world, we'd round up as many of these people as possible into a single room and create an "ISV's club" where they can harness the accrued knowledge and expertise of those in the community right now to get these bridges built.

and why should they care to do this? because if they don't, their software is going to keep arriving DOA on Linux. no matter how big the Linux desktop market grows they'll never be able to tap into it to its full potential unless the software quality improves. the ISVs who will make their mark in the maturing Open Source desktop market will be those who make the necessary investments. it won't take much, but without it they won't get much either.

we have a vested interested in this as well, because their software (if done well) will make our platforms much more interesting (and in some cases, possible to use at all) for the larger, global desktop market. it's also a potential well of new development talent and a lot of useful feedback.

we have two groups who have mutually beneficial motivations here. now the question is: can we make it work culturally?

users: scratch your itch

first it was Eugenia's rant. or rather, rants... plural.

then Maurizio Colucci started a thread on kde-devel and kde-usability about setting up a way for users to vote with dollars. this caused a huge cascade of discussion. not all of it sane.

in fact, it was in this conversation that Waldo got outed and quoted by Eugenia. this annoyed me quite a bit at first. the whole discussion put my teeth on edge.

but why? i enjoy working with people who use software i write. if it weren't for the other people in the community, developers and users, i don't think i'd write software. i'd probably go off and draw lines in sand boxes instead. so what was it that sat funny?

and then it occurred to me while discussing these things with Manyoso on IRC why ...

when Eugenia complains about developers, she's expecting them to do something about it.

Marrizio's ideas also put the onus on the developers to provide a fix. but.... this is something that users want. when a software developer wants a program to do something, they usually scratch that itch and do something about it. so why is it that when the users have a problem with the social scene, they don't try and scratch their itch? why don't they try and hack the social system?

i bet if these users put their heads together they could create some (social) system that would coax and encourage developers in the direction they'd like. and no, bitching on the internet doesn't count as a social system.

so all you disgruntled users: scratch your itch. come up with some sort of system that works for you and invite the developers to it. make it inviting. make it worth our while and your while. you're the only ones who know what you want. so hack the system to get it.

LUG Radio

was on LUG radio today. the hosts are pretty GNOME friendly *cough* *cough* it went well, though, i think. especially seeing as i was kind of out of it when they phoned me with a bit of a head cold.

well, looks like LUG Radio is unreachable right now. perhaps it's because both theDot and OS News have picked it up? cool beans.

speaking of being kind of out of it, i got two hours of sleep last night. in my chair at the office. i hate database migrations. at least this one was in the right direction: MS SQL to PostgreSQL.

KDE optimization files: katepart

i few months back i talked about konsole being a nice svelt little app. 50k for a new terminal tab is pretty good. but memory usage is only one of the "metrics of efficiency", another being speed. sometimes these two play a game of tug-of-war: to limit memory consumption an application might keep less information loaded at a time, relegating it to hitting disc or regenerating lots of data structures whenever data not in memory is needed and removing the data that was there previously. so, smaller memory footprint, but slower to do anything.

when it comes to editing text files, something i do a lot as a programmer, i care mostly about speed. enter the katepart: the text editor component that powers kate (surprise! ;), kwrite and kdevelop. katepart is heavier on memory than my other favourite editor, vim. while vim will be taking a couple of MB of RSS, kwrite will be chewing up 4-6MB. the difference decreases with larger files (as the text data becomes more and more of the actual RAM consumption), but the difference is still there. but that's ok: what i care about is speed. and boy, did i ever get a lesson in how fast katepart can be yesterday.

i had a 94.5MB file full of SQL statements. 1.25 million of them, actually. MS SQL server, which spat out the statements so i could stuff them into a nicer database, didn't enclose the date entries in quotes like it did the text entries. how very thoughtful of it. (side note: MS Enterprise Manager is a joke and an insult.)

so i crafted a regular expression of medium complexity, and verified it graphically using the amazing KDE RegExp Editor (that took exactly one mouse click), then took a deep breath and hit "Replace". three minutes and 3.75 million matches and replacements later, the file was in good shape.

that's in excess of 20,000 successful RE matches and replacements a second. colour me impressed. deeply impressed. oh, and did i mention that syntax highlighting was still on? see, now that's the kind of performance i look for in an editor.

ok, so i had a second file that needed the text ", False" removed from the end of each line. it had approximately 2.5 million records in it so i thought i'd edit it on the server instead of transfering it over the network, changing it and sending it back. the CPUs in the server i was working on were probably only 60-70% the speed of the desktop machine i was running kwrite on. so i figured to myself, "a bit more than half the horsepower, twice as much data, but the server is at 0 load and has more RAM and the search and replace is trivial compared to the date RE monster... let's see how vim fares!" i figured it would probably take 15-20 minutes at most to work its way through it. (honestly, i expected more like 10 minutes.) half an hour later i gave up and killed vim. a perl one-liner munched through it in good form, however, leaving me with the file i wanted in just a couple of minutes.

so to revisit the eternal question: is KDE bloated? well ... if you need to get a large file edited into shape, it seems katepart has some remarkable advantages over vim for certain workloads and vis is a console app. i'd say that's the opposite of bloat.

my hat is off to the katepart developers for a seriously fast and solid editor. thank you for making my pain slightly less painful yesterday.

HotNewStuff user interface

took some time this afternoon to look at the download interface for KHotNewStuff (KHNS). thinking about how to improve it, i came up with this as an early draft:

click to enlarge

Qt Designer file

a decent start? perhaps.

kio_httprc's needed

a bug was filed against konqueror today for a pair of websites that supposedly do not render correctly. but it turns out that the websites are buggy: bad javascript misdetects konqi and therefore feeds it junk HTML. switch the user agent and all is well. this is easy enough to do in konqi: go to the Tools menu, select Change Browser Identification, voila!

of course, the user who reported the bug still thinks konqi is broken as they didn't think to switch user agents, so all isn't well. this is an all too common event and results in unwarranted marring of konqueror's reputation.

so half an hour of putzing about later and i have a command line KDE/Qt app (weighing in at ~70LOC) called "agents" that rips through a kio_httprc file and spits all the domain-name to user-agent mappings into a postgresql database. this simple little app is aided by a short shell script that does some batch processing of kio_httprc files. sweet.

now i can do things like:

agents=# select count(id), d.id, d.domain 
                from domains d join agents a on d.id = a.domain 
                group by d.id, d.domain 
                order by count(id) desc;
 count | id  |         domain
-------+-----+------------------------
     2 | 179 | computerworld.com
     2 | 185 | sco.com
     1 | 177 | aec.at
     1 | 178 | coachella.com
     1 | 180 | marca.es
     1 | 181 | metica.se
     1 | 182 | msn.com
     1 | 183 | nickjr.com
     1 | 184 | pason.com
     1 | 186 | watchit.com
     1 | 187 | www.pcbanking.cibc.com

or

agents=# select distinct a.agent 
                from agents a join domains d on a.domain = d.id
                where d.domain = 'computerworld.com';
                       agent
----------------------------------------------------
 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)

awesome. not sure what to do with the data yet, but i have some ideas. like ... generate kio_httprc add on files for people to download containing the top N sites; country specific kio_httprc add ons; a way to quickly check konqueror bugs for specific domains for known user agent quirks .... sounds like a nice application for HotNewStuff, too.

but to make it all work really well ... i need you! well... not really you, but your kio_httprc file where all these mappings are kept. the more of these files that get fed into the database, the better. so if you'd like to help out email me your `kde-config --localprefix`/share/config/kio_httprc file and i'll add it to the database.

(for the privacy conscious, it's all anonymized in the database. and if it makes you feel any better, i'll trade you your kio_httprc for mine upon request ;)

.. everybody get together ...

a few months ago i noted how insanely great konsole was when it came to memory consumption, especially compared to launching a bunch of xterms. perhaps one would have thought that this good news would've been greeted with enthusiasm. or at least that people would go out and try and verify those results and be happy when they found out for themselves how efficient tabbing in konsole tends to be.

instead, the reaction i received was like a bucket of cold water in my face. many people, including some of our own core devels, decided to instead naval gaze about methodologies and poo-poo the idea that our software really was that great. granted, my observations were the digital equivalent of "back of the envelope" calculations, but they were generally accurate. it was a highly discouraging event for me, really.

ever wonder why people don't talk about KDE's efficiency? because we don't let them.

today i read in Ben Maurer's blog that Konsole takes "takes 50 kb per tab". he notes that even after Aivars Kalvans's great work on gnome-terminal, konsole is still 5 times more efficient with memory per new tab. now, how do you think he will be received by his community by announcing gnome-terminal's improvements? cheering? almost certainly.

will those same people who decided to get into a pedandist's duel when i said that konsole was so much better than xterm due to it's per-tab efficiency head off to correct Ben? probably not.

Come on you people now
Smile on your brother
Everybody get together
Try to love one another right now

it don't hurt to let someone else smile once in a while...

so here's today's cool note about something KDE's doing right IMHO: KIMPRoxy. the KDE instant messaging proxy provides an interface that any IM app can implement to publish information about who is online at the moment. in turn any other app can subscribe to this information and use it as it makes sense. so, for instance, kmail can show you that the person who sent you the email you are currently reading is online. whether that person is online in kopete, or kontact or any other such app. sweet.

instead of hardcoding interoperability between applications, which is short term thinking and brittle, KDE has built generic facilities for working together so that we may build applications out of smaller pieces of existing technology. in fact, this allows us to build new applications, even new types of applications, tomorrow without having to reinvent all the interoperability we've put in place everywhere else. this makes so much more sense than anything else out there and is really starting to show in our applications.

viva la 3.4!

trajectory

throwing things is art. a dance. a painting.

it leaves your being and from that point forward it's trajectory is out of your control. it goes where your motions really put it, rather than just where you wanted it to be. and the wind interferes.

most sports involve throwing things. with the hand, the foot, the head or a stick... but always with the mind.

there's a reason why they call the process of delivery in sales "the pitch".

KDE 3.4 has been tagged for release. 3.5 and 4.0 development will ensue.

next week i will be on LUG Radio, which is sort of like "the man show" meets "open source". tune in to see what hilarity ensues. i think i'll be the first KDE person on their show (though it was fun to hear jdub announce kubuntu this week on the show =). spread the word, get people listening. especially people who don't run KDE. yet.

btw, this week's show covered Hula as if it were something new and special. it isn't. it's Netmail, a second rate piece of software warmed over (sorry Novell =). not to mention it's at least the fourth such implementation in the open source world that i know of.

Kolab people: learn how to market yourself or suffer a disappointing irrelevance. Kolab is way more interesting and important than Netmail-cum-Hula, but NetMail-cum-Hula is getting marketed so that every git with two neurons is clucking about it, even though it's completely non-exciting given previous art and ultimately ill equipped to go up against MS Exchange. learn to appeal to the people with two neurons, because they outnumber by several orders of magnitude those with more than two. (ed.: wow. was i ever in a cynical mood!)

here's another good lesson: JWZ slams Netmail-cum-Hula, and the Novell people turn that into positive spin. that's how it's done! make lemon aide and people will drink the lemon juice. it may be frustrating that people are so easily duped, but that's life.

don't stop making better software, but learn how to sell it. it's not beneath you. unless you feel that having users is beneath you.

oh. and tune in to LUG Radio next week. yeah. *waves the flag*