I’m Christian (👋) and this is my bi-weekly (fortnightly?) newsletter with interesting content and links orbiting the world of graph.
Here we are at the end of 2020. I’m slipping in one more source/target before the end of the year. It’s that weird time between holidays so I’ve been hard at work on a fresh project.
Graphs
Back in source/target #17 I admired a newsletter called Winning the Internet built by Russell Goldenberg of The Pudding. As I wrote back then:
The idea of the newsletter is simple: they aggregate links from the most popular link-sharing newsletters—a daunting list of over 100 newsletters—and produce a breakdown of the most-shared links of the last 7 days.
This past week they released a dataset of over 100,000 links found in 113 newsletters since June 18th; wonderful snapshot of the last half of a pretty, uh, remarkable year. It gives us an exhaustive glimpse into the common articles and topics covered by the ever-expanding newsletter-sphere. A zeitgeist visualization, if you will.
Before the year was out, I wanted to take this dataset and build something to show off the interconnectivity of all these newsletters.
A source/target take on the “end of year” list.
After much data wrangling I present:
Instead of the ASCII Perl approach I shared in #17 I built a much more capable JavaScript app that allows you to explore the network and see the linked sites from each of the newsletters. I scraped additional metadata for each of the links and newsletters – you can see this when you hover or click on a node of interest.
This didn’t work for some of the links. Makes sense, it’s a lot of links!
I pre-compute my layout to minimize the amount of time and fan noise when opening the app. Saving these coordinates ahead of time this has the side effect of really ballooning the size of the graph on page load.
Another downside of this is that it takes a while to load at first: the loading animation is very necessary!
Project Processing
I’ve built a number of projects for source/target this year and my process is now pretty familiar:
- a rollercoaster of enthusiasm for the project …
- followed by a productive period …
- culminating in a tortuous mix of finalizing, tweaking and an (un)healthy dose of scope creep
It never ceases to surprise me that the last bullet point seems to take up the vast majority of time spent on a project. It’s a familiar observation that sits somewhere between the the ninety-ninety rule that states:
The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time.
and the Pareto principle that (roughly) suggests that 80% of my efforts will be spent on the last 20% of tasks. Some of this can be attributed to the use of new (or new-to-me) technologies for each project. I’m trying to build up a library of common code to use across projects but these things take time.
Over the year I’ve become progressively ruthless with project scope and continually reaffirm that done is better than perfect. In the case of “Half Year Hyperlinks” the result is an application that barely takes advantage of the underlying graph of links and newsletters.
I regularly tell people that it’s just plain hard to build interactive graph visualization applications. Challenges include a heady mix of data wrangling, interactivity design, performance considerations and the moving target of the modern web.
That’s not to say I don’t enjoy it–I’m clearly building these projects for some reason! I think source/target in 2021 will aim to build up a blueprint for the design and implementation of interesting, insightful graph experiences.
Until then, let me know what you think of Half Year Hyperlinks.
Links
Globular updates
I featured the new GitHub home page globe as part of #23’s “12 days of graphmas.” There’s now an interesting 5-part article series on how it was made.
Women at the top of the world, still not at the center
An article from The Brookings Institution on a paper using network analysis to measure the representation of different groups across 1,600 individual board members at nearly 100 globally-prominent organizations.
To establish the core of the network that is distinct from the periphery, we use an iterative process called k-coring, which identifies the group of organizational leaders within the network that are more connected to one another than the rest.
[We] find that women—and non-white women in particular—are proportionately more confined to peripheral positions within the network. We find that the representation of white males increases as we move from the periphery to the core. For the women, it is the opposite: Their representation declines as we move toward the core.
While it is perhaps not surprising that men still make up the majority of governing boards, what is significant is how being a woman disadvantages her from being included in the “core” of the network.
This is the network equivalent of not having a seat at the table.
Newsletter & Blog Recs
I like to read writing by developers who post regular updates on their work and process. It’s a great way to pick up new approaches to work and to help fend off imposter syndrome.
One of my favorites is Road to Ramen from DK. It’s a daily journal covering all aspects of being an independent developer working on new products to aid productivity. I love DK’s approach to product development, he guides clear vision into compelling apps and I’m excited to see his output in 2021.
I met Duncan Geere virtually at the formation of the Data Visualization Society last year. Duncan’s a thoughtful and inspiring information designer and I enjoy reading new editions of his newsletter chronicling his work and thoughts in 10 day increments.
Finally, Will Lyon (Neo4j) just started his own newsletter last month where he covers graph projects he’s working on. Will’s a very generous and capable graph advocate so I recommend subscribing here. For his latest project he’s building a Podcast Search app with Neo4j & GraphQL.
🎉 Happy new year to all Sourcerers throughout the world! See you in two weeks for #25.