April 2006

Today is the second anniversary of Beagle’s release to the world, and what better way to celebrate than with a release? Beagle 0.2.6 is out.

Now that Mono 1.1.13.5 — which has the fix for the leak I previously blogged about — has been out for a while, I decided it was time to back out my buggy workaround and just require it. It’s ironic that the workaround, when it failed, would cause Beagle to try to allocate huge swaths of memory. This release also fixes some problems when the beagle-crawl-system cronjob would stop working because some versions of su are finicky about the order of their command-line arguments, and the mysterious case of why some documents wouldn’t open when clicked on in beagle-search. (Turns out, it wouldn’t launch things from shell scripts.)
As I mentioned in the release announcement, Beagle wouldn’t be where it is today without the help of over 100 people who have supported and contributed to it over the past two years. I’d like to recognize those people in this undoubtedly incomplete list:

Jon Trowbridge, Nat Friedman, Miguel de Icaza and the Mono team, Daniel Drake, Fredrik Hedberg, Lukas Lipka, Debajyoti Bera, Veerapuram Varadhan, Dan Winship, Robert Love, Chris Lahey, Tuomas Kuosmanen, Ben Maurer, Brandon Hale, JP Rosevear, Luis Villa, Dave Camp, Joe Gasiorek, Anders Carlsson, Mikael Hallendal and Imendio, Rafael Slinckx, Larry Ewing, Alex Graveley, Christopher Orr, Jim Krehl, Garrett LeSage, Jakub Steiner, Vijay K. Nanjundaswamy, Adam Lofts, Adam Weinberger, Alexander Macdonald, Alexander Shopov, Amanpreet Singh Alam, Anthony Batchelor, Arangel Angov, Boris Peterbarg, Carl-Emil Lagerstedt, Chao-Hsiung Liao, Chris Schneider, Christian Laireiter, Christian Persch, Christian Rose, Christophe Merlet, Christopher Aillon, Clytie Siddall, Damodharan R, Daniel Naber, Daniel Nylander, David Richards, Dennis Cranston, Dennis Snell, Edward Cho, Enver Altin, Francisco Javier Fernandez, Frank Arnold, Funda Wang, Gabor Kelemen, Garrett Smith, Gary Ekker, Harish Krishnaswamy, Hendrik Brandt, Ignacio Casal Quinteiro, Ilkka Tuohela, Ivan Stojmirov, Jack Miller, Jakub Friedl, Jakub Steiner, James Ogley, James Vipond, Jean-Yves Lefort, Jonas Klingstedt, Jordi Mas i Hernandez, Josep Puigdemont, Joshua Nichols, Kang Jeong-Hee, Kevin Breit, Kevin Kubasik, Kjartan Maraas, Kostas Papadimas, Krzysztof Rosiński, Kwok-Koon Cheung, Kyle Ambroff, LaMont Jones, Larry Reaves, Lasse Bang Mikkelsen, Leonid Kanter, Lluís Pàmies, Luca Ferretti, Marijn Ros, Mario Manno, Martin Willemoes Hansen, Matthew Garrett, Max Wiehle, Maxim Dziumanenko, Michael Dominic K, Michal Kastelik, Miguel Cabrera, Nagappan Alagappan, Nicolas Trangez, Nikos Charonitakis, Øivind Hoel, Pat Double, Paul Betts, Raja R Harinath, Raphael Higino, Richard Dawe, Rich Midwinter, Robert Van Gorkom, Robinson Rodrigo Maure, Rodrigo Marcos Fombellida, Rodrigo Moya, Rodrigo Sancho Senosiain, Rostislav Raykov, Ryan Skadberg, Saravana Prabhu, Shobith Alva, Sivaiah Nallagatla, Sreekanth G, Srinivasa Ragavan, Stanislav Brabec, Stephen Solka, Subodh Soni, Sylvain Goletto, Takeshi Aihana, Terance Edward Sola, Tino Meinen, Todd Berman, Tom von Schwerdtner, Urko Fernandez, Uwe Hermann, Vaclav Slavik, Vincent van Adrighem, Vishravars Ramasubramanian, Vladimer Sichinava, Vladimir Petkov, Wojciech Polak, Wouter Bolsterlee, Yair Hershkovitz, Young-Ho Cha, Zafar, Žygimantas Beručka, Арангел Ангов

… as well as anyone I may have accidentally missed. Thanks! You’ve helped make Beagle the premier desktop search on Linux.

Good news everyone! Beagle is going to be participating in this year’s Summer of Code. Take a look at the Beagle SoC page for more information and a list of ideas. There is a wide variety to choose from: implementing a networked search infrastructure, getting involved with the nitty gritty details of Beagle’s internal indexer, profiling memory usage, improving Beagle’s user experience for NFS/Samba users, Firefox integration, Dashboard, and more. And if you have your own ideas, feel free to apply for those too, or discuss them on the dashboard-hackers mailing list.

Beagle 0.2.5 is out! This version fixes two big problems we’re seeing with 0.2.4: first, that it wouldn’t compile with Mono 1.1.14 and second, that web pages viewed with the Firefox extension couldn’t be searched. Also, there are a bunch of other bug fixes in there. 0.2.5 also has a fix for the X connection issues I mentioned previously.

This version allows the user to configure what backends to start up by default, and simplifies the sometimes confusing –allow-backend and –deny-backend options.

To see the current list of backends:

$ beagle-config daemon ListBackends
Allowed backends:
- KMail
- Files
- GaimLog
- IndexingService
- Tomboy
- Blam
- Liferea
- Akregator
- KonquerorHistory
- Kopete
- EvolutionDataServer
- EvolutionMail
- applications

Denied backends:
(none)

To deny backends, run beagle-config daemon DenyBackend name. To reenable it, run beagle-config daemon AllowBackend name. The next time you start the daemon, the changes will take effect.

In addition, you can tune this at runtime. To add a backend that was disabled in the configuration, you can use the –backend +name option. To prevent a backend from starting, use the –backend -name option; this is equivalent to the now-deprecated –deny-backend option. And to ignore the configuration completely and use only the backends you specify, use the –backend name option; this is equivalent to the also now-deprecated –allow-backend option.

Is there a term for words which spell the same thing on your phone keypad when you’re texting?  Like “have” and “gate” (4293), or “adds” and “beer” (2337)?  If not, I propose calling them numeronyms or perhaps telenumeronyms.

Please make checks payable to “Joe Shaw”.

Is the po/LINGUAS format set in stone? I ask because in SUSE we have to run gross sed scripts over all the configure.{ac,in} files to fix instances where the “no” locale hasn’t yet been switched to the “nb” locale. As the ordering and format of the ALL_LINGUAS field changes over time (usually to add linebreaks for easier readability), the sed script keeps breaking. If the po/LINGUAS file were one locale per line, this would simplify this greatly and we could just add this as a macro to run against all GNOME packages.

The script looks something like this:

sed "s/\(ALL_LINGUAS.*\) no /\1 nb /" configure.in > configure.in.tmp && \
mv configure.in.tmp configure.in

Ok, a follow-up to my Xlib post from yesterday.

To more accurately describe the situation, ideally what would happen when the X connection was broken is that we’d recover gracefully and continue on our merry way. How exactly we’d reconnect to X if it ever came back wasn’t something I had gotten around to thinking about. At the very least, exiting immediately without any shutdown is unacceptable.
A lot of people emailed me with suggestions, including even our own Havoc Pennington of Havoc Pennington fame! They basically fell into one of three categories:

1. Use an alternative X protocol implementation, like CVS Xlibs, XCB, or rolling my own minimal one.
2. Talk to the screensaver using D-BUS.
3. Use longjmp() to ensure that the handler method never returns to Xlib.

Unfortunately none of these is quite what I want. Here’s why:

1. This adds a new bleeding-edge dependency to Beagle, or requires that I write a new (and certainly buggy) X protocol implementation. Beagle is just now getting to the point where it isn’t a total pain to compile, and I’d really like to keep it that way. Plus, we do use the convenience of GDK to poke at X ourselves, and using an alternate system would likely cause conflicts. If we stopped using GDK, it would require us to do all the low-level X work ourselves, which would be annoying. All together, I don’t think this solution is reasonable.
2. This only works with gnome-screensaver. Its D-BUS interface is org.gnome.ScreenSaver, so we can pretty safely assume — at least for now — that no other screensaver program is implementing this interface. The KDE one probably doesn’t, and XScreensaver definitely doesn’t (and probably never will). Plus, our current approach works even if X simply blanks the screen. You don’t even need to be running a screensaver application. And lastly, we don’t depend on D-BUS right now, and the Mono D-BUS bindings are broken. At some point we will start using D-BUS for this sort of stuff, but things just aren’t quite ready yet.
3. This one is definitely the nastiest and a hack. Ultimately I don’t think it’ll work because this is a Mono app, and I have no idea what kind of side effect longjmp() would cause. And while it would mean that the function wouldn’t return, it would leave around all that X apparatus in a very unclean state. This means that somehow reconnecting later is probably right out. Everybody suggested this one, though. Are there actually people out there using this to some success? I’m really interested in hearing about it.

Even before I had originally posted, it was pretty clear to me that I would just have to deal with the situation. After posting, I played around with using XSetIOErrorHandler() to begin an orderly shutdown of the Beagle daemon. Our shutdown process essentially sets a flag and signals some threads to begin shutting down, and we block while this happens. Once all the workers are finished, we stop blocking and exit the main loop. So really, it’s ideally suited to this situation. I played around with it a bit and it seems to work pretty well. We do very little cleanup after the main loop exits normally, so we don’t miss out on anything important by letting Xlib exit for us. And really, it’s probably what you want anyway: most of the time Beagle is launched as a part of your X session, you want it to shut down when you log out. This half solution allows for this, but also lets you to run it successfully without any X session at all. I checked it in last night.

One thing I noticed when I was testing this was that our IndexHelper process, which is spawned by the daemon as needed to actually do the heavy lifting of indexing, was also making an X connection and dying when the X server went away. That process uses a main loop mainly for convenience (and gtk-sharp doesn’t have a way to do purely non-UI glib main loops); it should never connect to the X server. My fix for this? Unset the DISPLAY environment variable before calling Gtk.Application.InitCheck(). Yeah, a little gross, but it works.

Is it really true that apps can’t gracefully recover from losing their connection to the X server? The XSetIOErrorHandler() man page says:

The XSetIOErrorHandler sets the fatal I/O error handler. Xlib calls the program’s supplied error handler if any sort of system call error occurs (for example, the connection to the server was lost). This is assumed to be a fatal condition, and the called routine should not return. If the I/O error handler does return, the client process exits.

This sucks! The Beagle daemon doesn’t require X, but if it’s available it listens to the screensaver extension to see whether or not the session is idle and speeds up indexing if it is. But if the X connection is at any point lost, the daemon exits *immediately*. This doesn’t give us a chance to gracefully shut down, and there doesn’t appear to be anything we can do about it, short of moving this screensaver code out of process. (Which would be a real pain, and slow down indexing in general.)

Anyone have any suggestions?

There is a Firefox extension to let you view tooltips without them being ellipsized.

Stephan Binner has announced a 0.1 release of Kerry, a KDE frontend to Beagle. Along with Bera’s yaBi, we now have two great KDE clients. I hope that this brings even more community development to Beagle, and leads the way to tighter integration inside the KDE desktop. Kerry will be included in the upcoming SUSE Linux 10.1 release. Check out the screenshots, and download the source.

Beagle 0.2.4 is out. I think this release has the most contributions from outside (non-Novell) contributors than any other release, which I am thrilled about. Thanks so much to everyone who pitched in!

This release has a few nice new features, including GIF, XSLT, RPM and Gentoo Ebuild filters, big optimizations with the fallback sqlite databases and beagle-build-index, fixes to various crashes in the beagle-search UI, and still more memory fixes. As always, you can get more information and download it from the Beagle site.