The Creator of Curl Remembers 23 Wildly Successful Years
Daniel Stenberg is the original creator and long-time maintainer and lead developer of the ubiquitous cURL data transfer utility — which started life as 300 lines of C code and grew into over 172,000 lines of code in detailed data transferring libraries, used in billions of applications around the world.
This month, for curls’s 23rd anniversary, he looked back over a lifetime spent happily developing.
Writing from the municipality of Huddinge, just south of Stockholm, Sweden (where Stenberg does commercial curl support for a living), Stenberg observes in an anniversary blog post that libcurl “has over time developed into a de facto internet transfer API.”
“Today, at its 23rd birthday that is still mostly how I view the main focus of my work on curl and what I’m here to do,” he wrote.
After decades of refinement, its various libraries now support over 26 different transport protocols, and supports numerous customizations. (Just for example, curl can now be built with 43 different TLS libraries.) Its widely regarded status makes it the go-to component for a wide variety of applications which at some point in their lifecycle will need to reliably transfer data, and Stenberg writes that curl is bundled in top distros and operating systems (including Mac OS X and Windows 10, as well as iOS, Android and various Linux distros).
But that’s just the beginning, Stenberg notes. Several major apps (including YouTube, Skype, Instagram, Spotify and Netflix) each bundle its own separate build of curl, meaning many smartphones actually end up having multiple installations of curl. Curl is also used in game consoles like Nintendo Switch, Xbox and Sony PS5, and in top video games, including Grand Theft Auto V, Fortnite and Red Dead Redemption 2. It’s even in set-top boxes like Roku and Apple TV — so Stenberg estimates it’s in “maybe half a billion TVs.”
He’s also counted at least 22 different “smart cars” from major brands that are using libcurl to transfer data to and from the vehicle. You can also find cURL in Internet of Things devices, including printers, smart watches, media players and medical devices — so Stenberg estimates the total number of devices with cURL installed is over 10 billion.
Even before you consider that it’s also used in most internet servers, and is the default transfer engine for PHP…
But while taking a moment to acknowledge where it ultimately ended up, Stenberg also shared some fond glimpses into how it all began. And another short biography on his web page even shares his memories of discovering “the joy of computers” for the first time in the early 1980s on his friend’s Commodore 64. “I entered data sets in Basic that we eagerly read in some of the first C64 magazines at his place and since then I’ve been hooked.”
Portrait of a Young Programmer
Fourteen-year-old Stenberg and his younger brother Björn saved their pennies so they could buy their first Commodore 64 in 1985. But soon after learning the BASIC programming language, he’d realized all the cool stuff was being written in Assembly language, and joined in as part of the thriving demoscene subculture making short, slick audiovisual programs. “This was when I started spending spare time on programming. Up to several hours per day.
“This is something that I’ve never stopped doing since…”
Stenberg applied for college — until fate intervened, and he accepted a job at IBM instead. There he had his first exposure to Unix and the C programming language — learning mostly by reading the documenting man pages for various Unix tools — and eventually Stenberg moved on to becoming a full-time C developer, and later, an embedded systems consultant.
But, with a friend, he’d worked on building a text editor for the Amiga along with a dedicated, Emacs-inspired scripting language that he used to power a bot on IRC chat channels — a fateful project from which curl was ultimately born. Stenberg had wanted to let people query the bot to convert prices from one currency into another. He found the online tool ‘httpget’ (originally created in Brazil by Rafael Sagula) to perform periodic queries, and Stenberg remembers that in time he’d “more or less taken over as leader of the httpget project” — a project he renamed several times as its functionality evolved into the software he still maintains today. “Small and quick decisions done back then, that would later make a serious impact on and shape my life. curl has been one of my main hobbies ever since – and of course also a full-time job since a few years back now.”
While curl remained popular as a command-line tool, “I of course suspected that there would be programs and systems out there that could benefit from getting curl’s powers into their applications and that doing curl as a library would enable that.”
By 1998, its website was bragging that the tool had been downloaded “more than 300 times.” “I never had any world-conquering ideas or blue sky visions for the project and tool. I just wanted it to do Internet transfers good, fast and reliably and that’s what I worked on making reality.” Stenberg remembers the early days, when curl had just 2,200 lines of code and only seven contributors. Even after six years, the total number of contributors was less than a hundred.
But now he’s proudly presiding over a thriving community of volunteer developers — and over 2,300 people who’ve contributed code over the lifetime of the project.
Going Big
Around 2004 Stenberg applied for funding from the Internet Foundation in Sweden “to get some focused development time on curl,” and later received some funding from Adobe to bring SFTP support to curl. And it’s been going strong ever since, now supported with sponsorships and donations. (Although he recently retweeted what appears to be a visual metaphor.)
#opensource 👀 pic.twitter.com/spSLcNpPKs
— Wassim.js (@manekinekko) March 29, 2021
Every month the curl website now serves 15.6TB of data.
Over the years, there have been 2,348 contributors, Stenberg writes in his blog post — as well as 6,787 bug fixes. And all these years later, he remains grateful for the contributor community — and perhaps sees it as an echo of the values of early days as a programmer. “I had used so much open source and I wanted to be as cool as the other open source authors,” Stenberg remembers in a 2019 Stack Overflow post. “Thanks to it being open source, literally thousands of people have been able to help us out over the years and have improved the products, the documentation, the web site and just about every other detail around the project.”
Leading up to its 23rd anniversary, Stenberg tweeted out 23 different statistics about curl.
And earlier this month, Stenberg’s work was honored with a special tribute from GitHub. They mailed Stenberg his own GitHub contribution graph for 2020 — converted into a steel sculpture with a 3D printer. “Your hard work, care, and attention haven’t gone unnoticed,” they wrote in an accompanying letter, adding that the sculpture “tells a personal story only you can truly interpret…”
WebReduce
- Bell Labs remembers the other OS by Unix’s founders: Plan 9.
- For Women’s History month, GitHub shares stories highlighting female developers.
- Lexicographer using Archive.org assembles new online dictionary of science fiction terms (including “Robot”.)
- The Louvre puts its entire art collection online.
- Exploring the secrets of the world’s Tetris champions.
- When the U.S. Navy sponsored a hacking contest.
- After 52 years, a cryptographer and a programmer finally deciphered a message from the Zodiac Killer.
