pandas support may be coming in PyPedal 2.1

Posted on December 2, 2011 by jcole

I’ve been experimenting with pandas, a data analysis toolkit that provide an R-like data frame object. I am close to having working code that converts pedigree data into a DataFrame instance, which should make a lot of computations much easier. For example, some of the reports in the demographics module would probably be reduced to one-liners. It will also make it much easier to access advanced statistical functions in R using RPy2. This added functionality may not be particularly useful to someone who just wants to compute coefficients of inbreeding, but it may be very useful to other researchers. And to me, of course.

Posted in Python | Tagged , , , , , , | Leave a comment

Update: UniCurses on Windows

Posted on September 29, 2011 by jcole

A quick update to my earlier post about getting UniCurses working on Windows — for some reason, it works as described with Python 3.1, but not 2.5 (I know, I’m a little behind). I’ll get it sorted out one of these days.

Edit: You know, reading is a wonderful thing. It says right here that UniCurses requires v2.6.1 or higher. Good grief.

Posted in Articles | Tagged , | Leave a comment

Library Solutions for Homeschoolers

Posted on August 13, 2011 by jcole

Recently, Misty and I were talking about solutions to one of the perpetual problems faced by homeschoolers: keeping the library organized. (Hint: if you homeschool and do not have a library, you’re doing it wrong.) Anyway, I dug out my old PS/2 :CueCat and used it to add a few books to Tellico and Data Crow, which are Open Source desktop applications for managing collections. That worked nicely, but I ran into two challenges: 1) I would prefer that the data be accessible from pretty much anywhere, and 2) my laptop doesn’t have a PS/2 port, so I need a USB :CueCat (they do exist).

With respect to point 1, I have a web application that I’ve been writing and rewriting for years now that works okay, but doesn’t have any of the fancy JavaScripty/AJAXy bits that let it properly parse :CueCat input, so I’d have to add another feature when there already are so many that don’t work as intended, and I have so much free time.

While Google-ing around to address point 2, I found what is basically a turn-key solution for $40. Not super-cheap, but not bad that all. For that you can get an unlimited lifetime LibraryThing account — good grief, someone on there has over 45,000 books in their library — and a USB :CueCat for convenient data entry. I remembered that I had an account there already, which lets you enter up to 200 books, and confirmed that my PS/2 :CueCat works like a charm with the site. Thing One objected to the idea that he spend the next week carrying the books down from upstairs, scanning them, and putting them away, so a USB scanner that can be connected to Misty’s or my laptop seems to be a worthwhile investment.

I’m not saying that my time is worth millions, or that there is anything wrong with learning new skills, but for a pretty meager amount of money I can get everything organized pretty quickly with a minimal amount of work. LibraryThing is about 20 times cooler than my tool, even if I didn’t write it and it’s quite fast. You can’t beat that with a stick, I tell you what.

So, I think that we have a solution to our problem.

Posted in Articles | Tagged , , | Leave a comment

Lyx 2 and Sweave on Windows

Posted on June 12, 2011 by jcole

Just a quick not to say that I just got Lyx and Sweave working under Windows XP. Maybe I’ll post some links later, but for now I just wanted to say that it was pretty straightforward. I installed R 2.12, the current version of Rtools, and Lyx 2, as well as updating my MiKTeX installation. The only problem that I had was that I that I copied Sweave.sty into the wrong place in my C:\texmf tree. Once I got that sorted, it works like a charm.

Maybe I should add Sweave output to PyPedal…

Posted in Articles | Tagged , , , , | Leave a comment

New Tools Coming to PyPedal

Posted on February 25, 2011 by jcole

While I sit here on the train I thought I’d let you know what’s going on with PyPedal these days.

First, I do not have plans for a Python 3 port at the moment. One of the big blockers on that is matplotlib, but it is being ported. I’m not yet sure about the status of ReportLab and the rest of PyPedal’s dependencies.

Now, for the more exciting stuff, which will be in the 2.1 release. I’ve started working out the algorithms for some of the new tools I want to add, such as pedigree merging. Ideally, I can figure out how to do this with the __add__() method so that the syntax is easy: think “ped3 = ped1 + ped2″. I also have some ideas for what “subtraction” and “multiplication” of pedigrees might mean, but they’re a little sketchier right now. Ideally, I need a way to get a publication about this stuff to make it worth my very scarce time right now.

I plan to implement a faster algorithm that the recursive tabular method of VanRaden for calculating coefficients of inbreeding, but I’m not yet sure which one.

Just this morning I had a breakthrough on unknown parent groups, so look for that, too.

That’s enough to keep me pretty for a while, particularly if I am going to test everything reasonably well.

Is there anything you want that I’m leaving out?

Posted in Python | Tagged | Leave a comment

What’s the trick to getting UniCurses working on Windows? Promiscuity.

Posted on January 17, 2011 by jcole

You can’t really make a roguelike without curses, and while Windows makes many of us curse there’s not an official Windows port. There is, however, PDCurses. To keep us Python programmers sane there is UniCurses, “a wrapper for Python 2.x/3.x that provides a unified set of Curses functions on all platforms (MS Windows, Linux, and Mac OS X) with syntax close to that of the original NCurses.” Sounds nice.

It seems straightforward enough. You download and install UniCurses, and then you install PDCurses. I did that, and the test scripts didn’t work. The message displayed by the exception that was thrown indicated that PDCurses was not installed. Per the readme that comes with UniCurses, “The most common cause is the absence of the PDCurses dynamic link library in the program folder while running on Microsoft Windows.” This means that a copy of the pdcurses.dll file must be located in the directory from which you’re running the program. This gets a little trickier if you’re running IDLE on Windows to test your install — in that case, you must have a copy of the DLL in your root Python directory, e.g., C:\Python31. But once I got copes of pdcurses.dll spread liberally around Python directory tree everything worked just fine.

Note that if you’re running in a Unix or Mac environment than UniCurses uses the native curses library, and you don’t have to mess around with PDCurses.

Posted in Python | 1 Comment

I guess that nobody has been using that feature (2.0.1 bugfix release)

Posted on December 30, 2010 by jcole

Today I released PyPedal version 2.0.1, which includes a bugfix to the inbreeding routine. PyPedal will form a numerator relationship matrix (NRM) at the time the pedigree is loaded if you ask it to. and the NRM can then be used to find things such as coefficients of inbreeding. The inbreeding routine checks the pedigree to see if there is already an attached NRM, and if there is uses it rather than performing de novo calculations. There was an indexing bug in that part of the code that resulted in coefficients of inbreeding being assigned to the wrong animal. That has been fixed, and a stray debugging message also has been properly wrapped so that it won’t pop-up when it’s not supposed to.

Happy New Year!

Posted in Articles | Tagged , | Leave a comment

What is a secure homeland?

Posted on December 14, 2010 by jcole

So, I’ve been thinking about what I mean when I talk about homeland security, and apparently I mean something a lot different than a lot of other people.

Here are some things that I think lead to a secure country:

  • A robust economy in which people are able to find employment at a fair wage
  • Wealth shared fairly (not the same as equally)
  • A defense policy that secures our borders against foreign aggression, and does not engage in baseless wars, and which does not account for half of all discretionary Federal spending
  • Affordable educational opportunities.
  • A regulatory system which protects the population from corporate excesses.
  • Basic single-payer health care which can be supplemented with private insurance.
  • An end to the war on drugs, the accompanying loss of voting rights and civil liberties, and the prison-industrial complex..
  • An end to the death penalty — we should never trust a government with that kind of life-and-death decision.
  • A fair tax system in which those with the greatest need pay the least, and those with the most pay the most.
  • a completely overhauled intellectual property system which grants limited rights for short periods of time.
  • A self-sufficient food supply.
  • An program for energy independence that allows us to disentangle ourselves from the Middle East, and shifts subsidies from corn- and soybean-based fuels to cellulosic and microbial sources.
  • A national infrastructure that is modern and safe.

Here are some things that I think are harmful to our security:

  • The de facto suspension of Fourth Amendment rights for travellers.
  • Unchecked media consolidation, including Google and the telcos.
  • Free trade agreements that ship jobs and manufacturing overseas, transferring money from the employees to the executives, destroying jobs and communities.
  • Partisan hackery masquerading as responsible reporting.

I’m sure that I’ll think of more later. Now I need to pay attention to our speaker.

Posted in Congress | Leave a comment

Trying to make it here on Capitol Hill

Posted on December 6, 2010 by jcole

Today is day four of the orientation for my Fellowship, and we’re learning about Appropriation Committees and Congressional Oversight and Investigation. Exciting, I know.

What have I been up to? So far, I have spent two full and two half days in training, and I am searching for an assignment. Friday afternoon I dropped off resumes in several Senate offices (all Democrats, of course), as well as a Committee office. Looking on the House side of things will be a little trickier because of the swap in majority/minority status between the 111th and 112th Congress, but I do have a list of offices in which to leave materials. I also need to make follow-up phone calls to the offices I visited on Friday.

The job-hunting side of things is very stressful, and I hope it ends soon.

More later.

Posted in Congress | Leave a comment

Everyone said I was daft to build a castle on a swamp, but I built it all the same, just to show them.

Posted on September 29, 2010 by jcole

The day they said would never come has finally arrived — just a few minutes ago I tagged and released the first production version of PyPedal, my pedigree analysis software. This is the end of a seven-and-a-half-year development cycle, which included a Google-esque five-year beta period. The first dated entry in my 2,117-line CHANGES.txt file is from April 6, 2003, which is when I began the rewrite of PyPedal 1 (never released). There is a big gap in my log, with the next entry from March 31, 2004, relating to the handling of sires and dams with no pedigree file entry. Development was fitful, and you can see what particular things I was interested in at a given time by the clustering of topics in the changefile, which is probably pretty typical of open source projects with a very small user-base.

While I don’t know of many people using PyPedal I have benefited from the feedback of several users over the past several years, particularly Matthew Kelly, Dan Cieslak, and Matthieu Authier. A number of interesting features have been added because of their input, and some nasty bugs fixed, as well. I hope that people will continue to contribute bug report and feature requests.

So what’s next for PyPedal? Well, there will certainly be 2.0.x maintenance releases for bugfix purposes. There also will be a 2.1 development branch, which will focus on (in no particular order):

  • Some novel tools for manipulating pedigrees — extracting individuals or groups of animals, merging pedigrees, that kid of thing.
  • Better visualization tools, particularly for large pedigrees.
  • Improved documentation, particularly in the code itself.
  • Performance improvements, particularly in the inbreeding routines.

I guess that’s all I have to say for now. If you’re one of my users, thanks very much! If you’re not one of my users, there’s no time like the present to start using PyPedal.

Posted in Articles | Tagged | 4 Comments