Dougma (dŭg·mə) n.

Only private because I cannot afford the bandwidth of it being public until next month. (Used almost all of my 100GB limit this month…)

Program Committee Process (90mb flv file)

Writeup: TBD

I will be in Montana again in early July. This year I will have a rental car and thus more freedom than in the past. Are there any pythonistas in the Missoula/Lolo area? Or coding in general? Security?

It has become tradition that I start getting my personal game plan for PyCon together. I am hoping to do less coding and more organizational work. It would be nice to talk to anyone who has been to a recent conferences (technical or not, professional or not, python or not, does not matter).

Well no bears this time, and no pictures yet either. We had a fun time at our annual first camping of the year, and Sarah had a blast. The weather was perfect. Everything went great. Josh was a great big brother showing Sarah the ropes and introducing her to everyone. One of the highlights was playing tag with Josh, his friend Quit and my friends Matt and Deidra.

But for some reason I just did not relax. I didn’t get to spend as much time with friends or just vegging on the beach as I wanted, and I really have no one to blame but myself. Part of the problem is that I just could not shut my brain off. Last year I got a notebook, and it ended up being my ‘PyCon’ notebook. This year, I just didn’t seem to have time, oddly enough. Code freeze at work was Friday and that, I am sure, didn’t help. The official fork will most likely be this upcoming Friday.

I have had no time to work on any of my python projects, and it is driving me crazy. There are so many fantastic things happening with django-survey, and Pinax is picking up steam, I need to put the old PyCon ‘08 stuff into archival mode, and start up ‘09. There is some very very interesting stuff going on with the DFW Python group that I want to help out on as well. There are so many fantastic things being worked on right now by incredible people, and I feel a down right claustrophobic not being able to do anything myself. I only been able to attend one Boston Python Meetup so far this year!

It looks like I will be in Montanna the first week of July this year (my anual pre-PyCon-Tech kickoff-kickoff). Barring any project ‘issues‘ I hope to get my act together then (with respect to PyCon software for 2009). If there are any pythonistas in the Missoula area, please send me an e-mail! For now I guess I should get back to unpacking the car.

Steve Holden is participating in the 5K Race for Hope. He is looking for people to sponsor his run. Lets show the other groups the generosity of the Python Community! (Sorry Team Hopkins, Steve got to me first )

If Steve is willing to go the full 5K distance, we should be able to support him with some cash. With the exception of this past year, it has been his fund raising efforts which have kept PyCon so cheap. Lets put some of those saved pennies towards a great cause!

Well, this was going to be three or four posts, but thanks to some interesting announcement from google, it all sort of runs together.

It still will be I think. I will most likely try to rewrite things to give an overview and go into detail on specifics later. Things are getting interesting at work so we will see how much time I have to pull that off.

Files

Ivan beat me to the punch on the main gist post. While at PyCon I had the opportunity to chat with Mike Fletcher, another OLPC volunteer whom I forget their name, Phil Hassey, Richard Jones, Jeff Rush, and about 5 other people who wandered in and out of the small sprint room we were all half passed out in. People came and went durring the discussion (I believe Richard and Phil went off to play a board game at one point as well) which ranged from modern Sci-Fi offerings to games to global warming being a net win for Canada to the history of the world (not the movie). I should have gone to bed well before the discussion started. The discussion turned to the object store on the OLPC platform. Jeff, coming from a ZODB background, was quite pro object store systems replacing ‘file systems’. This is a hot button topic with me. This topic has come up at every professional job I have had going all the way back to when I was an CO-OP at Motorola as a ‘Document Administrator’ (secretary). In fact the only two topics which are more hot button for me are ‘common application UI framework’s, and ‘security after the fact‘. I first started thinking about this subject back in 93 when I first started working on  MUDDs (warcraft, only 100% text for you youngins). The world was editable online (like a lisp MUSH) but also had revision history (via RCS initially). We were dealing with ’serialization’ and how objects were managed. I fell in love with the idea that everything could be described as having a set of attributes (tags) and really you wanted to store and manage these things by those attributes. Permissions were nothing more than attributes. Actions were nothing more than attributes. Meta data by definition were just attributes. We struggled with systems for this, but I came away convinced that we needed a new paradigm in object storage, and this ‘file’ stuff was running on borrowed time. It came up again at Motorola for document management. It came up again at OpenVision (later Veritas) for backup and security compliance. It came up again with ClearCase and Derived Objects. It came up again with ‘dictionaries’ and data management for VoiceXpress. And the code base I currently work on has something called ‘DFiles’ which I can not discuss except to mention the name (DRAT!)

Storage

Back to the discussion at PyCon. I wish I had a transcript of the discussion (no I don’t… I was not as coherent as I think I was). The Idea that everything is just blobs in a cloud of data where the tags determine the meta-structured is nice, but there are some problems. The first and most obvious problem is that it does not integrate well with existing technology and libraries. Decades of software has been written with the concept of files. You can try a fake mapping, and try to integrate things, but it does not work well. Then there is the concept of ’sub-blobs’. That is each of the pieces of data could have sub parts. This maps well to your document which might have a chart or spreadsheet as part of it for instance. This can greatly simplify serialization, and you get all those nice blob store things. Your in-memory structure is your serialization structure. But in reality we already have this. They are called files and directories. It is simply (*cough*) an implementation detail dealing with the storage mapping. Ok, there is nothing simple about it, but we will come back to this. The argument then turned to the fact that you can’t have a blob show up in more than one directory. False. Those are called symlinks, but again that is an implementation detail. One of the biggest benefits of an object-store-as-filesystem is the ability to find and manage things not in a ridged tree structure which does not scale well in the average human brain (where did I put my (ssh) keys again?) But in practice it is just replacing one confusing arbitrary structure with another on some level as it’s usefulness is measured by the quality of the tags, attributes, and indelibility of the data.If you had those things well defined in a directory tree structure, then it works just as well (as google desktop search proves). A more subtile problem is that not all tags/attributes are created equal. It took a long time for my betters and practical experience to prove this to me. Many attributes are only useful to programs. These programatic tags are for relating data, validation, encoding, and the like. Most of the time these are auto generated or involve mathematical computations. They are never intended for human interpretation,but are none the less crucial for data management. You can try to predetermine the different types of these meta attributes, or just lump them together, but neither of these approaches are really tractable. Spend some time deep diving into the abuses of the windows registry and you begin to get an idea of the issues.

I know I am glossing over all the details, and not really giving any points the attention they deserve. I am not even properly quantifying the points. Issues of language are completly being skipped over (try describing what a ‘word’ is in your application; try again when that application deals with speech and natural language… how does that abstract into meaningful tags?) Oh well. The point is there must be a happy median. We should be able to have something which has a file system programatic interface, as well as a generic data store interface. The browsing of the data should be an abstraction. If this is implemented with a classic journaling file system or in a database should be an implementation detail at the filesystem level. Why invent a new abstraction layer which everyone must now implement against when we have a perfectly good one that everyone already does? A file by any other name still contains data. If this is such a good model, think about extending it to namespaces. The problems in software code management (which is just data on a very real level) for which namespaces were invented exist on the filesystem as well. Chew on that while you code with Matrix.Optimizer and Optimizer.Matrix.

Google

Google has an interesting take on all of this. All of their service (news, documents, reader, calendar, mail, blogger, etc) all have a file like data storage for the objects represented. They use folders/directories (really tags). The only restriction is that the folders are only one level deep. I do not care for this myself. I would love to be able to have a ‘people’ folder under my ‘python’ folder and have only those times tagged with both ‘python’ and ‘people’ under that ‘folder’. Maybe that is just me. I would not want these sub folder relations to be automatic. I would want control over the layout, but have the population automatic. But that is the only extension to their system I would like to see. Beyond that it just works. It works with both the object store model and the file/directory model. If only google would open up their API’s a bit more to include this system. On wait, they just did. You know if I had hit ‘publish’ on this post last evening when I first wrote most of this, I would have been ‘prophetic’ or at least ‘first post!’.

It’s not all hearts and ponies and sparkle (even if it is python and an abstraction layer on top of django to boot!!!) I have been holding off on posting this err… post until I could formulate a non-reactionary opinion on the entire Google Apps thing. I now have an opinion and it is much along the same lines as Duncan McGreggor. The issues I have are both similar and yet unique to his, and I will post on them separately.

Python, and Conference Software

This post is already too long,and my laptop battery is dying (no the charger is at work ). Those of you that I talked with at Pycon about UnConference hosting know what this is all about, and I told you so ;-). The last piece just fell into place. With that, good night

PyCon is now just a memory. I am still recovering from it, and there is an IRC meeting tomorrow to deal with all those little pieces left over and kick off PyCon 2009 work (YIKES!).  Originally I was going to blog each day of the conference, but of course, that didn’t happen once the sprints started.

The numbers for the people sprinting are all over the place. It was impossible to get an exact count as people swapped projects, arrived at odd hours, and didn’t all eat the free lunch. I will never forget the image of standing up after the sunday lightning talks when Brett asked ‘How many of you are staying for the sprints?’, and every hand in the room went up. More people stayed for the sprints than saw my lightning talk on saturday (using my memory and the view of the audience as an accurate scientific measure). David yelled out something like ‘we need more food’ and ran to tell the hotel to set more plates. We were expecting about 150 at most given the normal ~15% of attendance. The hotel count of people fed lunch on Monday was 270+. I know of 5 people who did not eat that lunch and instead went out for food. As it was corn beef and cabbage, I believe there were a number of vegetarians who were also unsatisfied and went elsewhere, but I do not have a count. Muddying the count is the fact that peoplefrom the hote, and not part of the conference, were also fed. I was in shock at how many people stuck around for multiple days.

The sprints were a huge success overall. I did not get to spend as much time as I would have like on teh PyCon-Tech stuff as I kept having to deal with a personal issue.  I apologize to those who took part in the PyCon-Tech sprint for not spending enough time with you guys. I had the opportunity to work with a great bunch of people, but between burnout, machine problems, and police reports my mind was sadly not 100% on the sprint. I will blog about the PyCon-Tech stuff later.

I have to say the best part of the conference as a whole was talking with all the amazing people. I got a chance to talk at length to James Tauber, which was a shear delight. It is amazing that it took going to chicago for us to spend so much time talking together, as we work about 3 miles apart. Talking with David Goodger, Steve Holden, Jeff Rush, Mary Rush, Brad Allen, and others was like coming home. I could name about 50 other people, but then things would get crazy. I can’t wait for Phil to come back to the east, and hopefully I will get a chance to program with him. His work blows my mind. I got to sit and talk with people I consider hero’s and anti-hero’s. I learned equally from both, and on a whole my life has been greatly enriched from the experience.

Finally I need to mention the people of ChiPy. These guys are fantastic! Carl, Ted, Steve, and everyone else are amazing amazing people! I am looking forward to working with them again; in 13 hours it would appear.

I should mention that the PyCon 2008 videos are now popping up on YouTube. I guess its time to add some more of my own.

First up is a quick peek at the now famous ‘Teach me Twisted’ talk that I shot when I realized I was witnessing something special.

Next up is a shot of lunch on Monday… in the word of Gossamer, ‘People!’

Finally, here is a peek into the Django sprint room Tuesday before lunch.

*For those interested in modern mono no aware literature I highly recommend the works of Hiroyuki Morioka; especially his Seikai (stars) series. The anime is supurb, but the novels are even better. I am still awaiting translations of his latest work. If not for PyCon I would have been going out of my mind looking for someone to translate it for me. If you are a Japanese translator, please contact me!!!

Well the Python lab seemed to go ell last night, and I had a lot of fun doing it again, even if the turnout was… well not what I had hoped. I think the people who did attend enjoyed it enough. Most people have come back to talk to me about it today. One person asked if I would be doing it again tonight, but alas, not.

I saw some of the most fantastic talks today. I could talk about those, but that is not what I will do.

Read the rest of this entry »

Today I am going to start blogging much earlier while I still have some brain cells left, and maybe I can include some links this time.

Right now the Lightning talks just started so this post might end up being just as scatter brained.

This morning I got up late (because the hotel alarm is quieter than Sarah when she sleeps), and rushed down stairs. I felt bad leaving AMK alone at the registration desk during the initial rush, especially as I had not shown him how to print badges or even where the badge equipment was. It turned out that we really didn’t ave any wireless network (again) so printing badges and doing registration was out of the question anyway. We have seen a HUGE spike in on-site registrations over past year. Last year we had something like 5-6 on-site registrations. I handled 12 yesterday and AMK handled another 10 (at least) today. This doesn’t even scale with the increase in attendance. It also does not include the additional vendor passes (which I have completely lost count of). The end result is we have approximately 1040 attendees on site. That is a 74% increase over last year. (more…)

Read the rest of this entry »

Well the Tutorial day is over and I am still alive, which I guess is a good sign. I woke up a bit latter than expected but made it down to the atrium for 7am to set up the on-site registration. Karl found an ancient beat-up laptop with no screws (literally!) which we hooked up a usb keyboard and mouse (because the keyboard and trackpad were not in the best of shape), and plugged into the wall as the wireless was still ‘flaky’ in the atrium. Oddly enough this plucky little beat up silver PII running unbuntu was working faster and doing heavy javascript live search on 1500 entries faster than my P4 800 laptop running XP! Things went smoothly and I handled about 10 people in long order. Long order in that most people who needed problems resolved were not of the easy kind, which was the story of reg in general this year. Many, many, many people wanted to change tutorials, or add tutorials. Everyone wanted to get into the most popular ones which were already over booked by 2-6 people (no suprize). There were some people whom I could accomidate and over the day I was able to fill up the PyGame tutorial and the other two that had open slots. Due to a no-show and a mutual swap I was able to get two people on the waiting list into tutorials they wanted as well. Originally I was going to flat out reject any tutorial changing, but that isn’t very ‘Python’…. Every break I had between tutorials, I would head back to the reg desk, and invariably there would be people waiting for me or looking for me to deal with reg issues. Mary got cross with me on a number of occasions for staying well past the end of the break, and missing parts of my tutorials. She also made sure I got some food in me, pretty much by force.

So I was a bit late to the OLPC tutorial, and by the time I got there, wireless down stairs was gone. It was PyCon2006 all over again. Wwhen I did a scan of the network I saw 12! adhock machines broadcasting the pycon SSID. They basicly took out the real access points. Some access points stopped broadcasting their SSID’s and people typed in the SSID on the sheets taped to the walls. the only problem is that windows will broadcast if they cant connect to an access point with that SSID. At one point the OLPC machines were blamed, but that seems like a cheap scape goat given some of the other information. in the end, wireless came back around 6pm…

The tutorials were a lot of fun. The scientific computing tutorial was good but it covered things at too high a level for my liking. I am regretting not going to the numpy tutorial after all. I have a feeling the level I was looking for was a bit more hands on than could be done for 50 people. I will try to catch up with the enthought guys later and dive into parallel differential equations and linear regression in the solver (there was one other thing that I was supposed to go over for work but now I forget what it was >.<). The testing tutorial was a blast, and it it always nice to see people talk about testing the same way we do in the MREC group. My focus was split in that tutorial as some of the topics made me thing of a javascript problem I was having with the schedule app. The end result is we now have reddit support directly in the app, and a reddit group ‘pycon2008‘. One problem with the app is the Dojo Toolkit. We are using an ancient version that has problems with subscripts in the tooltip iframes which do injection. The end result is I used selenium to grab the iframe that was being generated and stuck that in instead! Talk about direct application! (Someone in the PyCon IRC channel pointed me to the subgroup thing… thanks!)

I have some tabblo’s in the works from the pictures I took. They are uploaded to the site, but tabblo over the hotel wireless is less than functional. So instead I will just mention one very interesting piece of swag we received for the tote bags. I should mention that the tote bags are perfect for slinging your laptop in. Its easier than carrying around your laptop bag and swag and program guide, etc. Well we got 2000 little boxes that weighed a TON:

Lets take a look at the warning label for the item in these little boxes: