Loaded TAXACOM: A List about Biodiversity

A couple days ago we received a request from Roderic Page to load TAXACOM. He describes it as:

...a mailing list that dates back to the early '90's, and is a forum for taxonomists and other researchers interested in biodiversity. It is lively, with some long conversations. It's also featured as the source for sociological research, such as "Systematics as Cyberscience: Computers, Change, and Continuity in Science". Given interest in the Encyclopedia of Life (see also Ed Wilson's TED talk), which could be viewed as one response to the issues raised on TAXACOM posts over the years, I think it would be a very timely addition to MarkMail.

With a description like that, how could we resist! So I'm happy to say we've loaded the list, and (for trivia buffs) it even sets a new earliest list record in MarkMail, with archives starting back in 1992.

For more, see Rod's blog and email announcements.

Loaded OpenMoko: An Open Source Smartphone Platform

Last week we received on our feedback form a request to load the OpenMoko mailing lists. These folks are creating an open source smartphone platform, very cool stuff. Along with the request, the requester explained the benefit he saw in having MarkMail archive the OpenMoko lists, from the perspective of a project participant. I've reprinted it here with permission:

I would LOVE to see the OpenMoko lists get into MarkMail.. for 2 reasons..

1. From a developer perspective, I'm new to the OpenMoko platform and still learning the build system, etc. and am eager to start writing my own applications. But there's only so much info on the wiki and like many young communities all the juicy info is buried in the Mailing lists. So I'd love to be able to search all the lists for things like installing the sim card, what new hardware bugs they've found on the dev list, how to modify the dialer application, etc. This is where MarkMail really shines and is the best platform out there for this type of information gathering from community lists. If these lists were in MarkMail it would be one of the ONLY places one could find some of this information because of the advanced search functions in MarkMail. I think this holds true for a lot of young open source communities and MarkMail can really help out.

2. I think it would help the community in general by giving users an avenue to find the information they need to start better participating and contributing back. There would be less duplication of questions which distracts everyone on the list and hopefully more "I see how things are being done because I got all caught up by searching MarkMail, how about we do it this way.." etc.
Of course we loaded the lists for him.  Here's the activity chart:

Loaded Perforce: High-End Revision Control

Recently we loaded seven mailing lists dedicated to the Perforce SCM system. If you haven't heard of Perforce, they're a high-end revision control system, with a long list of corporate clients. They're known for speed and features.

I've been using a Perforce system to manage my own files for over a decade now, appreciating their free individual license.

Their mailing lists have a lot of technical Q&A discussion, so I hope having these lists more easily searched will help people find the answers they need. Here's the historic traffic pattern:

Loaded Eclipse and NetcoolUsers

Yesterday we loaded the email archive histories for two new communities: Eclipse and NetcoolUsers. Normally I wouldn't talk about these two communities in the same blog post, but after the load it occurred to me that both projects (coincidentally) relate to IBM. More about that at the end.

Eclipse (eclipse.org) is an extremely popular open source development tool project, initiated by IBM back in 2001. (The name was widely seen as an attack on Sun.) It took off and lots of Java developers use it as their IDE. They have a beautiful growth chart:


NetcoolUsers (netcoolusers.org) is a user community focused on IBM Tivoli Netcool. For a single list it's quite hopping (25 posts/day):


The fact both these communities relate to IBM is purely coincidental, but it's also interesting because it reflects the direction of pull we're seeing from the MarkMail user base. It's an early sign of what you can expect in the future: more technical content beyond open source.

Technical lists come in many flavors: pure open source (Apache, JDOM), corporate-sponsored open source (Eclipse, Xen), standards development (W3), technical user groups (NANOG), and groups focused on proprietary technology (NetcoolUsers). We plan to expand along each of these axes.

If you have a list you'd like us to load, let us hear about it.

Loaded NANOG: North American Network Operators' Group

Today we loaded a new list, NANOG, a discussion forum for the North American Network Operators' Group. In its 100,000 messages it holds some fascinating discussions about internet operations. The chatter around 9/11, Katrina, and y2k stand out especially.

The list extends back to April 1994, two months earlier than any list we previously loaded. It's always fun to break little records like that. It could be a while before we break this one again.


If you're intrigued by internet operations, the NANOG FAQ has lots of good factoids.

Loaded Python: A Cool Million Messages

Happy news: We've just finished loading the Python Software Foundation mailing lists. (Python is a popular programming language, overseen by the PSF.) With this load we're breaking a few records:

• Weighing in at 1,022,479 total messages, Python is now the largest community ever loaded since our initial launch. (We went live back in November with roughly 4 million Apache messages.)
• Half of those million mails are from a single list, python-list. That means python-list holds the new record for Crazy Huge What The Heck Can They Talk About So Much list. (And, would you believe, there's even more python-list histories from 1992-1995 still to load.)
  
• This puts our total combined MarkMail message count above 10,000,000. There was much hooting and hollering (and page refreshing) around here as the numbers clicked over.
• It's the biggest community ever loaded by our new hire Evan Paull. OK, it's the only community ever loaded by Evan. He started just a couple weeks ago. We figure after he's wrangled together a million message history, everything else will look easy.
  

Among the million mails are the archives for the Mailman project, something I'm especially happy about because much of our work here involves interfacing with Mailman, and this should help us understand it better.

As always, here's the traffic chart:

Loaded GNOME: 750,000 emails

Over the weekend we loaded the mailing list history for the GNOME project. GNOME is a immensely popular GNU project, a free software desktop environment and development framework. Their message traffic shows it has a vibrant and active community. They boast a history of 750,000 emails across more than 200 lists:

The peak in 2007? That's because in 2007 they started a new svn-commits-list (a list that captures emails about code check-ins) and it's been archived while the older cvs-commits-list wasn't. If we add -type:checkins to the query, we can graph the history without that list:

It took a fair amount of work to load the GNOME history because the archives had more spam and virus mails than could suitably be removed by hand. We had to use procmail and SpamAssassin to remove the junk.

One neat factoid: It's easier to remove spam from mail sent in 2004 than mail sent today. Spam blocking has always been a competitive arms race, but in this case we're fighting yesterday's war with today's technology! Even running in offline mode, SpamAssassin did a darn fine job.

I just wish it ran faster. If anyone out there is a SpamAssassin performance guru, please let us know.

Our thanks to Jeff Waugh for helping us get the histories.