fzf

fzf is a thing that everyone with a terminal should be using always.

Introducing Photodump

Photodump is an ultra-minimalist multiuser photo sharing application.  It's fast, it's free, and it doesn't require a sign up.

By Color

Apps arranged by color is the new books arranged by color.  Do it.  Get it done.

Back from the PNW

Back from a 10-day trip to Portland and Seattle.  It sure is nice this time of year!

Introducing window.js

Back at Sourcemap, we had a consistently inconsistent problem:  The shape of our tree diagrams could change dramatically depending on the type (and amount) of data being displayed.  This sometimes lead to the embarrassing problem of clients being unable to even find their data on the screen, much less make sense of it.

The solution was to increase the flexibility of our dashboard.  I'm a huge fan of tiling window managers, so I rushed out to make one in JavaScript.

The result was window.js, a tiling manager for web sites.  We hooked it up to our existing visualizations, did a bit of server-side magic to allow saving and loading, and voila-- 100% configurable, resizable, and sharable dashboards.

Check out the live demo here!

HOPEX Retrospective

I'm tired enough that a list of bullet points seems good enough:

• I don't like conferences very much.  I don't like amusement parks, Netflix, or night clubs.  I don't like packaged fun.  My fun must be difficult to contain.  I liked Hope.
• Hope and The Hotel Pennsylvania have a weird, magical chemistry.  The hacker ethic (especially of the networking and A/V team) was the perfect foil for the laissez-faire management.
• I was surprised how little coding happened.  There was plenty of show-and-tell, but not a lot of new stuff being worked on.
• Remember to bring your inflatable couch from home.  There are no guarantees that you'll be able to find one near Penn Station.

Introducing Sieve

In February, I set out to solve a problem with client-side data visualization:  How can I get data from the web into a format that my graph can use?

The usual answer is to scrape all of the relevant fields, parse them somehow, and store them in a database, then set up some interface for the graph to consume.  For a lot of projects, this makes a lot of sense-- But it introduces a lot of moving parts.  I started to wonder if a database was even necessary for small, one-off graphs.

I spent a few days mashing up Node, Request, and JSONSelect into a kind of proxy-plus-filter.  This meant I didn't have to worry about cross origin issues, and I could strip out unnecessary fields.  Sieve was born, and it was actually useful!

Over the next two months, I added the ability to combine multiple requests into one, stream results back via Websocket, cache, parse, and generally expand the list of probable use cases.  

Several months later, I've used Sieve for all sorts of things.  Not just data visualization, but all kinds of client-side applications.  Sieve has become phenomenally useful for me, and I suspect it might be useful for all kinds of web programmers.  Try it!  I'd love to know what you think.

Introducing Elemental

Elemental is a tool to quickly audit all the visual elements on a page.

This project came into fruition because I needed to refactor a large amount of CSS on a series of, well, complicated pages.

It works by loading the page in PhantomJS, rendering every element on a page to a PNG base64 string, streaming the results back via websocket, and sorting by visual similarity.

Be gentle!  It's still an early beta.

Code here.

Toggle Between Two and Four Spaces in Vim

To my fellow Vim users:

" Toggle between two and four spaces
let g:spaces = 2

function! TabToggle()
  if g:spaces == 2
    let g:spaces = 4
  else
    let g:spaces = 2
  endif
  echo "Switching to " . g:spaces . " spaces."
  let &tabstop     = g:spaces
  let &shiftwidth  = g:spaces
  let &softtabstop = g:spaces
endfunction

nmap <leader>t :call TabToggle()

Put that in your .vimrc and benefit.

Everything Driven Development

I see a lot of parallels with web development in the 2010s and financial services in the 1970s.  Both will be remembered as explosive, volatile industries with huge margins, questionable reputations, and colorful personalities.  Every innovation seems to have market-shifting implications:  Every new product, every trade, and every buyout has the potential to change everything.  Knowing the underlying reality of the industry is, as it was then, a guessing game.  For every Supply and Demand, there was a corresponding Stagflation, Bretton Woods, or Trickle-Down.

Like the early days of financial services, there is lively debate in the web development world over which methods and techniques are fundamentally correct.  Now, as then, it's hard to know which truisms will exist even a few years from now given the pace of change.  Once (https://martinfowler.com/articles/is-tdd-dead/) again (http://david.heinemeierhansson.com/2014/tdd-is-dead-long-live-testing.html), the web development community is awash in argument with no resolution in sight:  Heavily paraphrased, "should we do it this way or that way?"

My theory: _Everything_ works in our current environment.  We're early enough to the party that we don't yet know what all these innovations are even for.

In the 1970's, interest rates topped 20%.  A host of new financial products sprang up from the combination of good market conditions and new technology: Bond coupons, mutual funds, and real estate derivatives, and others.  Without too many exceptions, these products still exist, and people still invest in them as well a host of other products developed in the 80s, 90s, and 2000s.  Some of these ideas turned out to be more valuable than others.  However, instead of disrupting at every turn, businesses carved new niches as the industry grew.

Test Driven Development works because everything works.  Feature Driven Development works because everything works.  Our industry is growing, and these are all niches that will go on to employ people and create value.  Some techniques will turn out to be more popular than others, but there's rarely a reason to abandon one in favor of another.

I suspect that the reality of our industry will turn out to be just as messy as any other industry.  For now, there's a rising tide, and it's lifting all our ships.

Top 5 Consumerism Wins

Here are the best five things I've purchased on the Internet (non-referral links so you know I'm telling the truth):

Odormute.  Magically good odor neutralizer.  Far superior to febreeze, and much cheaper.

503TORCH.  Small, cheap, effective butane lighter.  You need a butane lighter.

Zojirushi Travel Mug.  Nondescript travel mug that doesn't leak, ever.

Makita Vacuum.  Replaced my old vacuum, roomba, and broom.

Good Grips Tea Ball.  Makes tea.  Tea is awesome.

The Best RSS Feed in a While

This might not be news, but if you haven't seen Spurious Correlations, I recommend it.  The RSS feed is consistently the best thing on my reader.

(Sorry, but I have to take some credit for it).

The things we miss

There's a hacker news comment that has stuck with me through the years:

"For a Linux user, you can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem. From Windows or Mac, this FTP account could be accessed through built-in software."

This was posted in 2007 regarding the fledgling Dropbox.com.

In the echo chamber of hacker culture, it's easy to dismiss new ideas for bad reasons.

Stop Trusting Startups

We have some exciting news that we wanted to share with you. Loom is now officially part of Dropbox, and we couldn’t be happier to join forces!

… 

As of today, we are no longer enrolling new users. Existing Loom users can continue to use our service until May 16, 2014.

When a service shuts down, it's not only bad for users.  It's bad for startups as a whole. 

 Why adopt a new service when it might not exist in a year?  Worse yet, why trust a group of people that will walk off with millions of dollars and leave you S.O.L.?  

I wonder if this-- not talent or user acquisition-- is the driving force behind some of the recent mega-buyouts.  It certainly benefits the Big Players to shake up confidence in small, independent companies.

And if you're not playing already, I encourage you to try the post-acquisition drinking game.

Lessons From App Week

App Week 2014 is over, and I failed.

But that's okay!  I have a nearly-completed codebase and a handful of valuable lessons:


1) Turn features into functions.

Before you even start coding, think about the actual functions you'll need to make a feature work.  That means plenty of up-front planning.  It also means really dialing into the libraries you plan to use.

For example, did you know that MongoDB can't do "$near" on polygons?  Neither did I!


2) With Cordova/PhoneGap, that hard parts are easy, but the easy parts are hard.

You'll be surprised how much time you spend shuffling around icon files and splash screens.  There are a handful of customizations that can make your life much easier.  For example, check out this post from Holly Schinsky.


3) The iOS simulator is old and broken.

It'll get you 90% of the way there, but don't lean on it for anything other than the most basic features.  You'll have to use real hardware if you want to test anything involving a compass or accelerometer.

App week: 7

(Faithful readers will notice that today's update is a little late.  Er... I meant "business days"?)

I'm realizing why MongoDB isn't the absolute perfect choice for every application in the universe.  And that reason is joins.

Some data just naturally lends itself to joins.  With the way it's laid out, GTFS is very well suited to doing joins.  Even the most popular MongoDB GTFS implementation uses Mongoose, so it's basically SQL.

I'm going to push on with MongoDB, although I can freely admit that it is a mistaken endeavor that should never be repeated.

Once I get my join problems solved, I look forward to submitting my app for review!

App week: 6

Lots of polish on the frontend.  New styles, assets, copy.  I suspect I'm going to need to integrate a map somehow if it's going to get approved.

Regardless, I'm looking forward to hooking everything up tomorrow.  I think it'll be pretty slick.

App week: 5

Home stretch!

I managed to check a few things off on the front end.  Most importantly, the icons, styling, and markup are all done.  Location services seems to be working.

One worry I have is that my app is absurdly simple in functionality.  I've read that Apple will reject apps that are too simple, which doesn't bode well for my short timetable.  I might have to look at shoehorning in a settings menu, interactive maps, or some other nonsense.

My experience with Cordova thus far has been positive, although there are a ton of obvious pain points.  For example, deploying an icon for your app-- It's probably one of the first things you do during the development process-- took me about an hour and a half because of all the sizing issues.

It would be easy to un-suck Cordova with some simple hooks and scripts.  The authors got pretty close, but it's nowhere near as smooth as, say, an Express install (although it does share much of the same technology).

How about a version of Cordova that's just the Javascript API and compliation tools?  Anyone?

App week: 4

It's come to my attention that APP WEEK 2014 is more than halfway over.

The backend is looking good-- Importing works, 2dsphere indexing works, and intersections work.  Still have a bit more work to do matching the shapes up to actual route numbers, but that shouldn't be bad.

I think the biggest challenge is going to be hooking up the realtime data and creating a notion of inbound/outbound.  This is tricky in Boston, because I suspect there are places where going "Inbound" actually takes you away from the city center.

The frontend?  Haven't started it.  Whoops!

App week: 3

Backend day!

Just finished up importing Boston's transit shapes into a Mongo GeoJSON collection.  That was harder than it should have been.

Tomorrow I'll be working on intersections.  Once I get that going, it's all about setting up an simple API with Express and building a simple frontend.