Posts Tagged mssql

Python for quick data processing

Lately I had a client ask me to work on a tool to check and email account, look up certain files XML which could be inside a zip file, then extract certain info, push that to the database.

Usually this client has all it’s software under .Net Framework, which under the standard library does not have a POP3 client library, you have to go for a 3rd party one, looking around for options I tough to myself “If I did this in python I would have easily done all that by now”, so yeah I gave it a shot, used poplib to access email, with email module to parse and extract files, ZipFile to compressed files and lxml.etree for XML processing, most of the modules are under the Python Standard Library and are well tested, in a few minutes I had the script done, now what was left was database access, GUI and reporting.

For the database they use a combination of Advantage DB and MS SQL Server, the table I needed to access was on the MSSQL Server, fortunately SQLAlchemy can work with it quite well, for GUI elements I used PySide which is working quite well right now on Windows.

The only thing missing now is Reporting, I would usually go for ReportLab + tRML and PDF output, however this being a Desktop application needed to be a bit different, I looked up at options with Qt (and PySide), NCReport looked nice but is a bit pricey and no Python bindings, most reporting tools are sort of standalone, you cannot plug them in your application and feed them custom data, thus applications such as JasperReports or Eclipse BIRT are out of the window.

Unfortunately there are no High-Level interfaces or toolkits for reporting under Qt + Python, which strikes me as a downside, having to work you way by hand is a no no at this level, it just wastes time and the results are usually not very reusable or flexible. I tried QtWebKit which could be almost perfect if it could do multi page printing easily, but no. I may end up just doing a simple report on VS.Net and launching a separate app for reporting.

Guess it’s time to roll our own Qt based reporting engine, which could be plausible, the QGraphics classes seems to be fit for this kind of job, but there are many missing pieces that would need to be fill in. Layout engine is sort of there, but I don’t see multi-page rendering support, you can render Qt widgets but it really looks out of place, kind of like printing a screen shot for a report.

In conclusion, I guess I could have done all that under C#, but I find Python to be more convenient for me (as I’m way more proefficient on it that on C# + .Net), guess a combination of the two would do for now.

, , , , ,

No Comments