Author Archive

Hackweek 31

November 25th, 2015 by Nathan Potter

What’s Hackweek Again?
Here at Chartbeat, we have a Hackweek every 7 weeks. Hackweek is a time to learn, explore, and just try something new. At the end of October, we had Hackweek 31 (that’s 31 x 7 = 7 months of hacking over 4 years). Lately we’ve been doing team hacks, which are 3 or more people doing a project together. This is a short writeup of just 4 of the projects that got done during our last Hackweek.

Team Gibson (Paul, Matt O, Anastasis)

Here’s a little known fact about AWS Redshift: it was previously known as the “Gibson” project at the Ellingson Mineral Company before that whole Hack the Planet / DaVinci virus scandal drove the company into the ground in 1995. Amazon acquired rights to the project in subsequent years and turned it into the data warehouse we know and love. To the disappointment of programmers everywhere, the product managers at Amazon decided somewhere along the way it would cut costs and ease adoption to remove the original graphical interface to the database and replace it with the Postgres-compatable one we have today.

Okay, not really. The Gibson was the supercomputer featured in the 1995 teen sci-fi thriller, and now cult-classic, Hackers. The movie energetically explored and sensationalized the early Hacker subculture and effectively inspired a whole generation of kids to become computer programmers. Myself (Paul Kiernan) and two other initiates of the hackerverse (Matt Owen and Anastasis Germanidis) came together over the past hackweek at Chartbeat to make the original Gibson interface a reality and perhaps even bring it to the Oculus Rift.

We began by investigating the Oculus Rift SDK and frameworks we could use to include an existing C++ project that emulated the Gibson interface. Unfortunately, it quickly became clear we would be unlikely to produce anything in a reasonable amount of time if we had to struggle with the lack of support for the SDK on OS X, learn how to use openframeworks, and refactor the existing project to include Redshift adapters and work generally with VR in c++. So we researched alternative platforms and settled on the widely accessible combination of WebVR and threejs.

WebVR is an experimental Javascript API that provides access to Virtual Reality devices like the Oculus Rift. Combined with threejs, a wonderful javascript framework around WebGL, we were able to create a Gibson-like interface to one of our Redshift clusters that runs entirely in the browser. The final product queries our cluster to get a list of tables and their sizes and paints them as buildings on a 3D landscape. Buildings are arranged randomly on a plane and their heights are a function of the number of rows in a table they represent.

Here you can see two videos of the gibson interface hooked up to one of our production Redshift databases (with table names obfuscated for, you know, security reasons).

2D Gibson: https://youtu.be/t5PbktJ8YBU

3D Gibson: https://youtu.be/UCUC2aIroJI

The Gibson project can be hooked up to any Redshift cluster so let us know if you’d be interested in playing around with it!

Hack the planet!

real hacks
-paulynomial (Paul Kiernan)

Team Faceblocker (Dvd, Nate, Immanuel, Jess)

With all the recent uproar about ad blocking, the Faceblocker team went to work figuring out (a) how to make an ad blocking extension of our own and (b) how to replace ads with something even better! We wanted to put a little soul back into the ads… to help make advertising personal and relevant. So we decided to jack into the user’s webcam.

Faceblocker is a chrome extension that replaces ads with YOUR FACE. Unfortunately, after using it for a couple days, we started getting a little tired of staring at ourselves in 300x250s, so we cooked up a couple different options, including: Darude’s Sandstorm and this inspirational music video. Finally, we experimented with a chat-roulette style video streaming server that replaces each ad with a random feed of another user’s webcam who is also using the extension. A little creepy, but endlessly entertaining when you get enough folks using it.

– Dvd (David)

Team Mobimon (Alex, Tom, Jeremy, Mike)

Team Mobimon crafted the game Mobimon. We focused on the peer to peer aspect of the game’s original intention, which lead to a play on limited time turn based games. The stack behind Mobimon was React with redux, and Firebase. React was chosen for view management, Redux for state management, and Firebase was the crutch we relied on for pvp communication. The team, being all front end developers and a designer, decided to use Firebase because of the ease of use. We didn’t really use Firebase to persist data, but more for its websockets. We had the host player start the game and via websockets would push to Firebase which would then sync any other players with the current state and commands.

The stack was something new to all of us but through pair programming, we were all eventually able to go from crawling to running and contributing. We learned a ton, especially about redux’s strengths and its annoyances; let us not talk about the boilerplate around setting a simple http request. Overall, the game turned out awesome; in a week we not only learned about the whole new stack, a new implementation pattern, but produced something tangible with it.

– Alex

Team Confidence Interval UI (Ashley, Brian, Jenn)

Lately I’ve been interested in ‘explorable explanations’. Working with Brian Tice and Jenn Creighton from the frontend team, and Dan Valente and Josh Schwartz from the data science team, we decided to demonstrate the effect of sample size on the type of information you could reliably take from a normal distribution of data.

A good portion of the week was dedicated to first understanding the topic ourselves. The goals for the design were to use common English as much as possible, to interactively visualize the data being graphed, and present visualizations inline with their descriptions when possible.

A somewhat unexpected hurdle was using common English, as many words we use to talk about data vary in meaning from their technical definitions. Another challenge was finding an analogy that allowed us to cover the points we thought were necessary to understanding the concept.

Our solution used a narrative set in space (a standard science fiction strategy, as well as personally amusing), wherein the user imagines themselves a tailor tasked with outfitting an unknown population. Given a certain population size, interested in taking the minimum reliable number of samples, and aiming to match the current sizes in production, the user adjusts ranges setting the breadth of sizes available (translating to the range of the distribution and the target mean), as well as the sample size (creating a distribution that over time will match the target distribution.) Both ranges have small graphic representations next to them, the complete graph is displayed below, and all three adjust in real time.

– Ashley

Why Hackweek?
Some of the other projects that got done that week included “Team 404 – Death of the Shark” (a new interactive 404 page), “Team Clojure API’s” (rewrite APIs in Clojure, just for fun), “Apple TV Big Board” (a Chartbeat Big Board for Apple TV) and others. As you can see, we take our Hacking seriously. It’s an important part of our culture of learning, growth, and making great products. Come hack with us!

Why Open Source?

July 17th, 2015 by Nathan Potter

Rick Mangi pontificates on the open source efforts going on during Hackweek at Chartbeat.

Building an Entire Product in 6 Weeks

June 30th, 2015 by Nathan Potter

Harry Wolff, our intrepid Frontend Platform Engineer, blogged about the creation of the Paid Content tool.

Engineering Ladders – Helping Engineers Along Their Way

June 5th, 2015 by Nathan Potter

Not too long ago, Camille Fournier at Rent the Runway shared the software engineering ladder their team uses for promoting engineers within the development team. I thought I’d take the opportunity to share the ladders we use at Chartbeat and look at how you might structure a ladder for your startup.

First, The Basics.
A ladder is an outline of expectations for people at various levels within the company. Many companies of varying sizes have them, you may vaguely remember seeing one during a new hire orientation, or you may have revisited it regularly if you’re gunning to climb the ladder quickly. They serve both as a means for people to know what’s expected of them to get a promotion, and a framework for managers to know when to promote someone.

This is Chartbeat’s ladder for Backend Engineers. We have a separate ladder for Frontend Engineers and Designers, and another ladder for Data Scientists. For comparison, here is Rent the Runway’s ladder and Joel Spolsky’s ladder for Fog Creek.

I hadn’t seen Camille’s or Joel’s prior to doing ours, yet they have a lot in common. I guess there are some universal truths in charting growth once you know a little about how engineers and engineering teams work.

Do I really need a ladder? HR is such a drag.
If your company is just starting out, you probably don’t need a ladder. However, it’s reasonable for an engineer to start thinking about promotion eventually, and you’ll need to address that. You may be able to start by giving promotions in a more haphazard manner, but as the team grows you need to do it evenly and with some justification.
If your company is more than 2 years old, and you have employees that have been around since the beginning (and I hope you do) it’s time to start thinking about it. I joined Chartbeat when it was just over 2, but it had had an extended gestation at Betaworks and was just getting an established engineering team. I waited 3 years to put a ladder in place, which was probably a bit longer than I should have.

It’s also worth noting that raises are not tied directly to the ladder at here at Chartbeat. A move up the ladder will always come with a significant raise and title change, but smaller raises can happen in the mean time.

Two Paths
The Chartbeat and RtR ladders both include a concept of “Manager” and “Architect” tracks. This is a common distinction in software engineering teams and one that most developers will face at some point in their careers. “Do I want to build bigger and better systems, or do I want to manage bigger and better teams?”

Both jobs require leadership. For an architect, it’s more about thought leadership. Thinking long term and getting a team to share your vision. For a manager, it’s more about hiring, team organization, and helping people up the ladder.

There’s no right answer to this, it’s all about your personality and preferences. They are not typically mutually exclusive either. In my experience, if you’re excellent at one of them you’d probably be excellent at the other as well. Maybe this is why Fog Creek makes no distinction between the two (in their ladder, anyway).

Experience Counts
The Chartbeat ladder does not explicitly include years of experience in the requirements for moving up. However, I’ve told the team to expect it to take 2-3 years to move up a stage. RtR has what looks like years of experience associated with each stage on the left side of their Ladder. Fog Creek disassociates experience from skills entirely, but does tie it to salary.

How experience relates to the value you bring to a team as an engineer is not always obvious. In one regard, it could be argued that a new grad with all the skills of a developer with 25 years of experience should be paid the same, but are they really the same? What metrics do you use for measuring experience and how is it valued?

For starters, the top rungs of the ladder will include some things new grads will not have done at a company of scale. “Successful platform cycle launch”, “Plays a key role in developing multi-year technology strategy”, or “Owns the development for a major project” are not something you will have done without a few years under your belt.

But more to the point, if you are an experienced engineer bringing your hard earned knowledge to bear on your work; sharing experiences, offering insight and guidance, you are delivering far more than quality code to team.

Climbing The Ladder
When it comes down to it, creating a ladder is the easy part. How you apply it is much trickier. You can use a form, have a meeting, write a letter, but these are only useful if they create honest discussions around performance. An honest discussion, structured around the framework you’ve provided your team, is the best and most transparent way to help someone grow their career. Those discussions can be hard to have, but a ladder will help you focus on tangible actions both good and bad.

As a an engineering manager it’s your job to help your team up the ladder. That’s not just because it’s good for your business, it’s your commitment to them when they joined your team. If I didn’t believe that I’d have stuck with writing code instead of managing coders. I could have avoided a lot of meetings, but I like to think I’m helping good engineers along their way.

How designers can create interactive prototypes with Illustrator

February 19th, 2014 by Nathan Potter

On his blog, tomgermeau.com, Tom explains how you can quickly go from Illustrator to an interactive clickable prototype in the browser without the need for a developer, without writing HTML, in about 10 minutes.

Hacking Cassandra

January 10th, 2014 by Nathan Potter

Tadas (@tadasv) looked at implementing HyperLogLog in Cassandra during hack-week. Checkout his blog post on hacking cassandra.

Chartbeat Engineering Blog