I've been working on Datasette, a Python application for exploring databases for a few years now. Today I released a new tool built on top of it: Datasette Lite, which runs the existing server-side Python application entirely in your browser, using a WebAssembly version of Python. [more inside]
I've been exploring a way of running web scrapers for a few years that I call Git scraping: the idea is to scrape a source of data (like a "current power outages" map from an electricity company) into a Git repository such that the history of that repository tells the story of changes to that information over time. git-history is a new command-line utility I've released which helps convert that collected history into a SQLite database to support analysis using my Datasette tool.
I wrote a little utility to convert the massive text file one obtains from the Export Your Comments page into a variety of other formats suitable for various purposes. Currently converts to HTML, JSON, or Unix-style MBOX (mailbox) format. [more inside]
This is the new Mac desktop version of my Datasette application, which helps people explore SQLite databases and CSV files and install plugins to visualize them, clean them up and more. [more inside]
I was fiddling with a little personal project and needed to start writing some GUI code. GUI code is a pain in the butt to write and I really just wanted to get on with my project. So, obviously, I spent over a week writing a whole package to wrap Python's default GUI package, tkinter. As you do. [more inside]
I've been teaching and writing about data analysis and visualization using Python for a few years now, and an absence of teaching work during the events of this year has finally given me the time to put together a book. There are also articles about various aspects of data processing with the scientific Python stack (pandas/numpy/matplotlib/seaborn) and more to come.
CircuitPython is a version of Python for tiny computers called microcontrollers. We're celebrating CircuitPython Day on 9/9, the snakiest day of the year (or at least, that's what it says here), with livestreams and more. [more inside]
"The Art of Python" was a miniature arts festival on Friday, May 3rd, at PyCon North America 2019, focusing on narrative, performance, and visual art "that helps us share our emotionally charged experiences of programming (particularly in Python)." It featured 4 short plays, a song, and a video remix. I started and chaired the festival. Then I wrote up a retrospective with photos, discussing why I started "The Art of Python", what led up to it, and how I feel about its future. Since I cannot be one of the co-organizers for "The Art of Python" at PyCon North America in 2020, I’ve also prepared a HOWTO guide for people who want to do this sort of thing in the future. [more inside]
At the PyGotham 2018 tech conference, Jason Owen and I presented "Python Grab Bag: A Set of Short Plays", inspired by the Neo-Futurists' show "The Infinite Wrench". The 40-minute video is up on YouTube and my blog post links to the script and slides, credits the crew and cast, deep-links to the specific timecodes for individual plays, and gives citations for the references we made. [more inside]
A Twitterbot that posts freely-licensed photographs of clocks. [more inside]
What does critical data science add to our understanding of sexual harassment in academia? [more inside]
Just finished building a content recommendation engine for MeFi using natural language processing and non-negative matrix factorization techniques! It produces a list of post recommendations based on a user history of posts, comments and favorites. It can also make recommendations based on a piece of text, so for example, you could paste a particular post and it will return a list of other posts that have some similar characteristics. I hope you enjoy playing around with it! Please let me know what you think. Here's more info in case you're interested (: https://github.com/tomasbielskis/metafilterpostrecommender
Your team’s code review practices cause ripple effects far into the future. In this play, see several ways a single code review can go, then fast-forward and rewind to see the effects – on codebase and culture – of different code review approaches. Video recording of a 22-minute play about code review, mentorship, tech management, and regret. Premiered on October 6th, 2017 at PyGotham in New York City. Announcement blog post ("it's Run Lola Run but about code review") and wrapup post (audience responses included "I used to be that reviewer and I'm trying not to be anymore" and "I don't code at all but this is a marvelous management parable").
A few months ago I asked this question about replacements for reddit. Since then, I've been trying my hand at Python and ended up writing this program that pulls the top posts from your subreddits and emails them to you. This has worked well for me in that it allows me to bypass all of the muck and keep any reddit surfing to a minimum. [more inside]
@BardVs45 uses the 1990s era Shakespearean insult generator to respond to tweets from Trump. [more inside]
Monolog is an interactive diary bot that prompts you with interesting questions, which it chooses based on the topics you write about. [more inside]
Magically translate AskMe song recommendations into Spotify playlists (it takes a minute or two). [more inside]
You know that thing that happens three or four times a day when you need a random name for something, say a new soft drink or a recently discovered virus strain, so you say "I know, I'll use a ship name from Iain Banks' brilliant Culture novels!", and you have to google it, and eventually find the list on wikipedia which is fine, but you really just want a name, or maybe 2 or 3 names so you can pick, and not to have to read through a whole wikipedia page? [more inside]
Have you ever wanted to quickly find the largest files within a complex directory, but have trouble remembering the syntax for the
findcommand? This might come in handy. [more inside]
A twitter bot that uses machine learning to define invented words, posting truncated definitions on Twitter and complete ones on Tumblr. Tweet @lexiconjure a made-up word, and it'll define it for you. [more inside]
Neuralsnap generates an image caption using a model I trained (convolutional and recurrent neural networks), then uses another character-level recurrent neural net that I trained on ~40 MB of poetry to expand the caption into a poem. (In this example, generated from a Rothko painting, the red text is the direct image caption, and the rest is the poetic expansion.) [more inside]
'Alphabet Aerobics' is a pretty interesting track. I made it decidedly less interesting using some python signal processing libraries to make the whole track a constant tempo. [more inside]
My own humble contribution to the learn-to-program literature.
With Clarifai, an image concept extraction API utilizing convolutional neural networks, and ConceptNet, a lexical relationship database, I built a template system to generate paragraphs of text from photographs. word.camera is responsive — it works on desktop, tablet, and mobile devices. The code behind it is open source and available on GitHub, because lexography is for everyone. [more inside]
Opinion polls are all well and good, but they don't give you much of an idea of what might actually happen in an election (particularly in a multi-party democracy like the UK). Electobot aims to solve that by running thousands of simulated elections in order to work out what might happen if the election were run tomorrow with the polls as they are. In addition to running the simulations, I've also been blogging the results at Electobot: The Blog. [more inside]
Using Python 2.7 and the Natural Language Toolkit, I created a program called Sonnetizer that generates 14-line rhyming sonnets in (mostly) iambic pentameter from any text corpus. Using Sonnetizer, I generated 10,000 unique sonnets from the sonnets of William Shakespeare, and compiled them into a PDF.
I wrote a little library for working with functors, applicative functors and monads in Python. You can grab it from the Python Package Index. [more inside]
In the course of the recent end-of-year ritual of going through my past year's unfinished Ableton Live projects, I found myself writing a Python script to partially automate the task, by examining the Live sets and dumping information about them (namely, the tracks and the instrument/effect devices in them) as text. [more inside]
A course in programming in Python for literate non-programmers, offered in Brooklyn, NY. [more inside]
I have started a new technical blog, where I will be documenting various projects I have worked on (typically involving code, though not always). [more inside]
Anagramatron searches for anagrammatical pairs of tweets, the cream of which are retweeted and posted to an associated tumblr. Source on github.
GRARchiver is a small python script that archives all your Google Reader read items. You need to know how to use python to use it. [more inside]
Metropho.rs is a geographic metaphor map that plots "X is the Y of Z" tweets by putting the "Y" label on the "X" location. Some nice coverage by the Atlantic Cities blog here. [more inside]
Spotmaps is an on-going project to map the colour footprints of different films. The website was designed to show off the completed image library. [more inside]
We started out as Rails developers trying to script capistrano deployments and somehow ended up hacking together a full platform-as-a-service system, run on dedicated servers, with S3-compatible distributed storage and a REST API/build process that's mostly heroku-compatible (i.e. Procfiles and buildpacks and one-time-processes). We've got support for Ruby 1.9.3 and 2.0 preview including Rails, node.js/CoffeeScript, python, php, etc etc. [more inside]
A general-purpose dataflow programming language based on Python, written in Python [more inside]
Kicksaver finds Kickstarter projects that will end soon without meeting their funding goals. If you have a few dollars to spare, you can help tip a struggling project over its goal and prevent it from losing the pledges made so far. It also tweets endangered projects a few times a day. This is a weekend project I made as part of my continuing effort to teach myself programming. It's completely noncommercial, open source, and not affiliated with Kickstarter in any way (although I do think they're pretty great). [more inside]
My friends and I built an 80s Joke Line for a hackathon event here in Toronto. You can call in to hear a joke and then share one of your own. It's cobbled together using Flask (a Python web framework) and APIs from Twilio and Soundcloud. The code is on GitHub. We are a bit short on jokes. Maybe you can help with that. [more inside]
Ever wanted to know where in Oregon the next new bar will be? Every week, Oregon's liquor control commission posts pdf's of their applicants for a liquor license. This bot grabs the latest PDF, makes a google map from it, and then tweets it. [more inside]
This is a static blog compiler that works well with Git. [more inside]
Here is my video offering solutions to the global food crisis as part of a submission for the international Thought For Food challenge. Possible NSFW if your boss can't abide a male nipple. [more inside]
It's never too soon for more Markov chains, right? cobe is a Markov based text generation framework that behaves like the venerable MegaHAL chatbot, backed with a more modern (SQLite) datastore. Its brain is kept entirely on disk, uses little memory, and scales reliably. A very minimal IRC client is included. [more inside]