bdunagan

fill the void

Dollar Clock 4.0

Dollar Clock 4.0, now available.

Update: Testing releases is always good, as is removing debug code. I clearly did neither. On launch, the app ignores the stored time and sets the time to 01:06:40. I’ve submitted an update to fix this annoying issue.

Remind Me Later 1.4

Remind Me Later 1.4 is now live on the Mac App Store and this blog! This release is long overdue, so thanks for everyone’s patience. Many people contributed excellent suggestions over the past months, and I included quite a few. In particularly, thanks to the BusyMac folks for helping me integrate with BusyCal better. While there are many new features in 1.4, let me single out three of them: calendars, alarms, and todos.

Type in a [calendar]: The most requested feature was calendars. You can now specify any calendar as part of the event text, like “Buy groceries at Whole Foods [Home] 4pm tomorrow”, and Remind Me Later will add an event in the “Home” calendar with “Buy groceries” as a title and “Whole Foods” as a location at 4pm tomorrow. If you don’t specify a calendar, the app defaults to the calendar selected in Preferences, and the

Type in a @1h alarm: Along with typing calendars, you can also add alarms. When you type “@1h”, Remind Me Later will add an alarm for one hour before that event, and the dialog will show “1h alarm” next to the “Cancel” button. You can also type “@15m” for fifteen minutes and “@2d” for two days, anything of the form “@N{m,h,d}”. In addition, you can specify the type of alarm (sound or email) to add in the Preferences.

Type in a “todo”: Remind Me Later now supports To Dos. Just type in “todo” or “task”, and the app will add a task to that calendar. Moreover, if you use Things, you can turn on iCal syncing in Things’s Preferences.

Below is a list of all the new features in 1.4:

• Type in a [calendar]
• Type in an alarm like @15m, @1h, or @3d
• Type in a “todo”
• Better BusyCal integration
• Things integration with iCal To Dos
• Full set of alarm preferences
• Unclutter the menu bar by hiding the icon
• Menu bar icon correctly inverts when clicked
• Global hotkey works correctly for Shift, Control, Option, Command keys
• Support for “+N{m,h,d}” delays
• Specify a default event duration in Preferences
• Lion (10.7) compatible

Download this new release from the Mac App Store or this blog!

Visualizing 144,000 Minutes in iTunes

Digging around Things’s To Do list inspired me to tackle a larger personal dataset: iTunes. One feature I love about iTunes is play counts. The app keeps a cumulative count of how many times I’ve played any given track. These counts are maintained by my iPod and iPhone as well, so the numbers give an accurate picture of my music history. Combining those counts with track lengths, I can see how I spend my days in iTunes.

I wrote a Ruby script using Nokogiri to parse ~/Music/iTunes/iTunes Music Library.xml, extracting out tracks with their time and play counts using SAX callbacks. Then I used the excellent d3 to visualize the data as a squarified treemap.

Here’s what I came up with: 2,700 tracks played, 28,000 plays, and …

100 days in iTunes is about 10% of my life, given that my iTunes Library file is three-years-old. Hans Zimmer’s scores to “The Dark Knight” and “Inception” represent nearly 20 days, and a single song fills 10 days or 1% of my life. I guess variety kills my focus.

Also interesting is how much I listen to podcasts:

• 5 days of The Economist
• 64 hours of Scotty’s various Cocoa developer productions
• 19 hours of Daniel Jalkut and Manton Reece’s “Core Intuition”
• 12 hours of Jared Spool’s “UIE”

Googling around, I found remarkably few people delving into iTunes as a dataset:

• Niem Lam’s Parsing iTunes for Processing
• Quora’s “iTunes Analytics Dashboard” question
• No Sleep’s Super Analyzer
• iTunes in ManyEyes
• Last.fm’s LastGraph and LastHistory

I did stumble upon Planetary for iPad. The free app visualizes your iTunes library as a universe, where each artist is a solar system, each album is a planet, and each song is a moon. The app varies the size of the moons based on play counts. It’s a fantastic app from Bloom, a company founded by people from Stamen.

I’ve included the Ruby source code below. It reads in the iTunes XML file and outputs a JSON file with the track information and an HTML file with the d3 Javascript code embedded.

# Require libraries
require 'rubygems'
require 'nokogiri'
require 'open-uri'
require 'time'
require 'json'

# Handy extension
class Object
  def valid?
    !self.nil? && self.length > 0
  end
end

# Simple object for Tracks
class Track
  attr_accessor :title
  attr_accessor :artist
  attr_accessor :album
  attr_accessor :play_count
  attr_accessor :length

  def valid?
    self.title.valid? && self.play_count.valid? && self.length.valid?
  end

  def to_s
    puts "[#{self.artist}] '#{self.title}' in '#{self.album}': #{self.play_count} (#{self.length.to_i / 1000})"
  end

  def total_seconds_played
    (self.play_count.to_i * self.length.to_i) / 1000
  end
end

# Handler for XML SAX callbacks
class TrackXMLHandler < Nokogiri::XML::SAX::Document
  attr_accessor :previous_item
  attr_accessor :track
  attr_accessor :tracks

  # Nokogiri delegate hook
  def characters(string)
    # Store previous valid Track.
    if self.previous_item == "Track ID"
      puts "#{self.track.to_s}" if self.track.valid?
      (self.tracks << self.track) if self.track.valid?
      self.track = Track.new
    end
    # Process Track attributes.
    self.track.title = string if self.previous_item == "Name"
    self.track.artist = string if self.previous_item == "Artist"
    self.track.album = string if self.previous_item == "Album"
    self.track.play_count = string if self.previous_item == "Play Count"
    self.track.length = string if self.previous_item == "Total Time"
    self.previous_item = string
  end
end

# Process iTunes library into Tracks.
trackHandler = TrackXMLHandler.new
trackHandler.tracks = []
parser = Nokogiri::XML::SAX::Parser.new(trackHandler)
# We only read the iTunes XML file, but still, run at your own risk. 
parser.parse_file(File.expand_path("~/Music/iTunes/iTunes\ Music\ Library.xml"))

# Iterate over Tracks.
total_time = 0
total_plays = 0
hash = {}
artist_hash = {}
trackHandler.tracks.each do |track|
  # Convert Track objects into JSON for d3 treemap (artist => album => title => total length).
  hash[track.artist] = {} if !hash.keys.include?(track.artist)
  hash[track.artist][track.album] = {} if !hash[track.artist].keys.include?(track.album)
  hash[track.artist][track.album][track.title] = track.total_seconds_played
  # Aggregate artist play time.
  artist_hash[track.artist] = 0 if !artist_hash.keys.include?(track.artist)
  artist_hash[track.artist] += track.total_seconds_played
  # Aggregate total time and plays.
  total_time += track.total_seconds_played
  total_plays += track.play_count.to_i
end

# Display basic stats.
artist_hash.keys.sort_by{|key| artist_hash[key]}.each {|artist| puts "#{(artist_hash[artist] / 60).to_s.rjust(6)} minutes: #{artist}"}
puts "=> #{trackHandler.tracks.count} tracks"
puts "=> #{total_plays} plays"
puts "=> #{total_time / 60} minutes"

# Write out JSON file for d3 Javascript in HTML file to read.
f = File.new("itunes.json","w")
f.write(JSON.pretty_generate({"itunes"=>hash}))
f.close

# Write out the HTML to make it work.
html = <<EOF
<html>
	<!-- HTML copied from mbostock's http://bl.ocks.org/972398 -->
  <head>
    <script type="text/javascript" src="d3.js"></script>
    <script type="text/javascript" src="d3.layout.js"></script>
    <style type="text/css">
      rect {
        fill: none;
        stroke: #fff;
      }
      text {
        font: 10px sans-serif;
      }
    </style>
  </head>
  <body>
    <script type="text/javascript">

// Increase the size for large libraries (2k+)
var w = 520,
    h = 520,
    color = d3.scale.category20c();

var treemap = d3.layout.treemap()
    .size([w + 1, h + 1])
    .children(function(d) { return isNaN(d.value) ? d3.entries(d.value) : null; })
    .value(function(d) { return d.value; })
    .sticky(true);

var svg = d3.select("body").append("svg:svg")
    .style("width", w)
    .style("height", h)
  .append("svg:g")
    .attr("transform", "translate(-.5,-.5)");

d3.json("itunes.json", function(json) {
  var cell = svg.data(d3.entries(json)).selectAll("g")
      .data(treemap)
    .enter().append("svg:g")
      .attr("class", "cell")
      .attr("transform", function(d) { return "translate(" + d.x + "," + d.y + ")"; });

  cell.append("svg:rect")
      .attr("width", function(d) { return d.dx; })
      .attr("height", function(d) { return d.dy; })
      .style("fill", function(d) { return d.children ? color(d.data.key) : null; });

//   cell.append("svg:text")
//       .attr("x", function(d) { return d.dx / 2; })
//       .attr("y", function(d) { return d.dy / 2; })
//       .attr("dy", ".35em")
//       .attr("text-anchor", "middle")
//       .text(function(d) { return d.children ? null : d.data.key; });
});
    </script>
  </body>
</html>
EOF
f=File.new("itunes.html","w")
f.write(html)
f.close

This script is also available in my bdunagan GitHub repository.

Developer Tip: Remap Caps Lock to Command (⌘)

Apple makes it easy to remap certain keys. The left Command key on my MacBook Pro’s keyboard never worked well ergonomically, so I remapped the Caps Lock (⇪) key to Command (⌘). I’ve been happy with it ever since.

Frankly, I physically removed the left Command key after that to prevent my muscle memory from using it. That worked quite well.

Visualizing GTD with 1.8K To-Dos in Things

I was poking around ~/Library/Application Support/ on my Mac and stumbled across the XML database for Cultured Code’s Things. Apparently, the first to-do I entered in Things was about Things.

While I’ve emptied the trash quite a few times since I began using Things in 2008, I still have 1.8K to-dos in the app, mostly in the logbook. I thought it would be interesting to plot them by month, using gRaphael.

Each month is a bar, and each January is a red bar. While I average around 50 to-dos each month, I added three times that number in July 2009, undoubtedly wedding tasks. Below is the Ruby script I wrote to process Things’s XML library and write out an HTML page with embedded gRaphael instructions:

# Require libraries
require 'rubygems'
require 'nokogiri'
require 'open-uri'
require 'time'

# Use Nokogiri and Xpath magic.
doc = Nokogiri::XML(open(File.expand_path("~/Library/Application Support/Cultured Code/Things/Database.xml")))
dates = doc.xpath("//object/attribute[@name='datecreated']").collect {|item| (Time.utc(2001,1,1) + item.content.to_f).strftime("%Y-%m-01") }
# Add up months and separate years (for gray/red bars).
counts_hash = {}
counts_months = []
counts_years = []
dates.each do |date|
  counts_hash[date] = 0 if !counts_hash.include?(date)
  counts_hash[date] += 1
end
counts_hash.keys.sort.each do |date_key|
  date = Time.parse(date_key)
  counts_months << ((date.month == 1) ? 0 : counts_hash[date_key])
  counts_years << ((date.month == 1) ? counts_hash[date_key] : 0)
end

# Write out the HTML and Javascript to make it work.
html = <<EOF
<html><head>
<script type='text/javascript' src='raphael-min.js'></script>
<script type='text/javascript' src='g.raphael-min.js'></script>
<script type='text/javascript' src='g.bar-min.js'></script>
</head><body>
<script type="text/javascript">
window.onload = function () {
// Add hover functions.
var fin = function () { this.flag = r.g.popup(this.bar.x, this.bar.y, this.bar.value || "0").insertBefore(this); };
var fout = function () { this.flag.animate({opacity: 0}, 60, function () {this.remove();}); };
// Graph with gRaphael.
var r = Raphael("thedata");
var chart = r.g.barchart(10, 10, 450, 100, [#{counts_months}, #{counts_years}], {stacked: true});
chart.bars[0].attr({"fill": "#666"});
chart.bars[1].attr({"fill": "#CD0000"});
chart.hover(fin, fout);
}
</script>
<div id="thedata" class="jsgraph"></div>
</body></html>
EOF
f=File.new("things.html","w")
f.write(html)
f.close

This script is also available through my GitHub repository.