StatusCode Weekly

GitHub's robot coding companion

Wed, 30 Jun 2021 00:00:00 +0000

#323 — June 30, 2021

👋 This is the last issue you'll receive for a while. StatusCode is taking a break. I'm not entirely sure how, if or when it will return, but there are so many stories to tell, links to share, and fantastic readers too, so I'll be back somehow but StatusCode will be taking a break for at least a month or two. Thanks for your support and, hopefully, I'll be in touch again soon.
__
Peter Cooper, your editor
(@peterc to follow me on Twitter – a good way to find out whatever's happening next)

StatusCode Weekly

What's happening in software development, ops, platforms and tools.

GitHub Copilot: An AI 'Pair Programmer' — A project that GitHub has been working on in the background for some time with OpenAI. It’s a VS Code extension that uses machine learning to suggest useful code snippets as you type and it works with lots of different languages. Everyone I’ve seen has a lot of mixed feelings about this, but it’s undoubtedly an interesting development. Maybe I need to create a newsletter version of this tool.. 😆

GitHub

Tributes Pour In For Near, The Developer Behind One Of The World's Best SNES Emulators — A person called Near created BSNES and essentially became a developer to spend years working on the ‘perfect’ translation of a Japanese RPG they loved. This may not seem like a developer story on the surface, but this person pushed forward the state of the art in emulation and even went so far as to hiding PC components in their luggage as a teenager to be able to scratch their development itch.

Nintendo Life

Infrastructure as Code with Pulumi - Any Cloud, Any Language — Build, deploy, and manage modern cloud applications and infrastructure using familiar tools and programming languages like Python, Go, JavaScript, TypeScript, & C#. Harness the power of software engineering practices for the cloud. Get started for free.

Pulumi sponsor

rfc.fyi: Search for RFCs, Fast — When they say ‘fast’ they’re not lying. This site is amazingly nimble at helping you track down the particular RFC you want, whether you want to read the FTP spec, a protocol for controlling coffee pots, or how QUIC works.

rfc.fyi

IN BRIEF:

Last week Microsoft unveiled the next version of Windows would be called Windows 11.
Forget about the hypothermia or deafness caused by hanging out too long in the datacenter.. robots are increasingly being used for datacenter maintenance.
In today's dystopic news, Ars Technica reports that Amazon is allegedly using algorithms 'with little human intervention' to fire workers.
If you had just one shot to redesign a part of the Internet, what would you change?
Martin Fowler is stepping back from speaking in future: "I far prefer a trip to the dentist to have fillings done".
DigitalOcean has extended their range of managed database systems with Managed MongoDB joining Redis, MySQL, and Postgres.

Ask HN: Do You Self-Host Your Database? — I’ve seen the self host vs managed database debate all over the place recently, so it was neat to see Hacker News take it on. Given the audience it’s perhaps unsurprising the majority expressed a preference for maintaining full control.

Hacker News

▶ Building Apps on the Decentralized Web with Nader Dabit — Jeremy Daly chats with Nader Dabit, a well known developer in the AWS and JavaScript worlds, about decentralized protocols, the ‘Web3’ movement, and what sorts of applications they enable. This is an area that continues to mystify me, so was a well timed episode :-)

Serverless Chats Podcast podcast

Change Data Capture at DeviantArt — How we made a streaming data replication solution from sharded MySQL to Apache Iceberg using Debezium.

Wix Engineering sponsor

Hosted Monitoring: Evaluating InfluxDB Cloud and Grafana Cloud — Experiences from someone who’s worked on self-hosted Grafana and InfluxDB setups but has explored the potential of managed cloud solutions.

PQVST

Giving Your Web Publications 'LaTeX Style' — A quick guide to using VS Code for creating documents for the Web with the academic LaTeX look and feel.

Stefan Gössner

Remote Repositories Extension for Visual Studio Code — If you use VS Code, this extension from GitHub itself offers a quicker way to open GitHub repos remotely direct from the editor.

Microsoft

El Carro: A Way to Run Oracle Databases on Kubernetes — Coming from Google, rather than Oracle itself, is an Oracle operator for Kubernetes.

Google Cloud Platform

Why Tracing Might Replace (Almost) All Logging

Lightstep sponsor

How to Migrate DynamoDB Tables from One AWS Account to Another with AWS Data Pipeline
Sodabathina and Gong (AWS)

When It Comes to Git History, Less Is More
Stephen Brennan

# Diving Into The Queue

Whenever I discover something I think would be a good fit for StatusCode, I add it to a queue of items to review. Due to the limitations of email (and your attention), a lot of good things never make it out of the queue and into a real issue.

It seemed fitting, then, for me to take a look over the queue and see if anything jumped out to share in this issue before we take a break. Here's some of what caught my eye:

STORIES AND TUTORIALS:

How I Hijacked The Top-Level Domain of a Sovereign State — This tale of how an engineer temporariliy took over 50% of all DNS traffic for the TLD of the Democratic Republic of Congo had StatusCode feature written all over it, but it still didn’t make it in :-)

Fredrik N. Almroth

Security Headers Quick Reference — This article lists and shares basic explanations of some useful HTTP security headers, including Content-Security-Policy and Cross-Origin-Opener-Policy.

Eiji Kitamura and Artur Janc

curl vs wget — Both popular tools when it comes to making HTTP requests, and I didn’t realize curl’s creator also contributes to wget too.

Daniel Stenberg

How to Protect Domains That Don’t Send Email — The British government has some technical tips for ensuring (or hoping to ensure, at least) that domains (which don’t need their own email services running) can’t be so easily used for spoofing.

GOV.UK

Create Animated GIF and WebP From Videos Using FFmpeg — Being able to cast the right incantations to get good results out of FFmpeg also impresses me. I have to keep articles like this bookmarked to figure it out afresh every time.

Matt Joseph

A Teenager's Guide to Avoiding Actual Work — How in 1982, the author successfully hacked his way out of having to fill in potholes.

Mad Ned

Making a 'Digital Clock' in Google Sheets — Spreadsheet tomfoolery never fails to amuse me. I’ve seen a lot of such tricks over the years, but never bothered to dig into quite how they work till now..

Robin Lord

A tcpdump Tutorial with Examples — 50 Ways to Isolate Traffic

Daniel Miessler

TOOLS:

Search 1 Million Linux Kernel Commit Messages — This is a fun demo of a fuzzy search engine called Typesense but trained specifically on commit messages for the Linux kernel.

TypeSense

Crontab.guru: The cron Schedule Expression Editor — The cron job scheduling format is something I’ve used for 25 years and still haven’t memorized perfectly, so tools like this can provide a handy reminder. There are others, if you prefer.

Cronitor

Slowfil.es: Deliberately Slow HTTP Resources for Testing — We’re all told to make our sites as fast as possible and there are numerous ways to improve the experience when page elements like images or fonts are slow to load, but if you want to test what happens for real.. this service could help introduce some true sluggishness into your system.

Harry Roberts and Ryan Townsend

Admittedly a bit of a random grab bag of stuff, but I enjoyed them all, and look forward to one or two of them making it to Hacker News later today, perhaps.. 😄

Software estimation is hard, but do it anyway

Wed, 23 Jun 2021 00:00:00 +0000

#322 — June 23, 2021

Web Version

👋 As part of our winding down to our July hiatus, last week we included a roundup of some of the most popular reads from the past few years. One even made it back on to Hacker News the same day, so it's great to see them resonate :-) This week we're running a roundup of the most popular 'tiny tools' we've linked to in recent years, so I hope you enjoy it.
__
Peter Cooper, your editor
(@peterc to follow me on Twitter)

StatusCode Weekly

What's happening in software development, ops, platforms and tools.

The 2021 Software Developer Shortage Is Coming? — A big headline from the ACM here, but the argument is that travel embargoes and significant declines in 2021 comp sci student graduation rates coupled with increased competition will only make a tight labor market even tighter. (Better conditions and more pay can always help though..)

Breaux and Moritz

Introducing AWS Step Functions Workflow Studio — AWS Step Functions has been around for years and is a serverless orchestrator that lets you build scalable apps using state machines. Workflow Studio takes things to another level providing a low-code visual way to bring things together. Perfect for anyone new to or otherwise intimidated by the usual process.

Marcia Villalba (AWS)

Advanced Engineering Go, Docker & K8’s Training — Ardan Labs provides instructor-led live-streaming training and self-paced online video training for teams and individuals. Join the 10,000 + engineers who’ve dramatically improved their engineering skills using Ardan Labs training now!

Ardan Labs sponsor

Benchmarking JavaScript Memory Usage — Historically, memory usage hasn’t been a heavily tracked thing in the JavaScript world, whether for a lack of tooling or motivation. This is changing and new tools and APIs are turning up all the time. Here, Tim has done some analysis of what the top 10k web sites are using in terms of memory and, perhaps controversially, how the choice of framework can affect things.

Tim Kadlec (WebPageTest)

IN BRIEF:

Estimating how long it'll take to complete a project or even to implement a single feature is hard but esteemed engineer Jacob Kaplan-Moss explains while estimation is hard, we should do it anyway.
I've been seeing praise for the Julia programming language (commonly used for numeric analysis) bubbling up lately, more recently in "My view of Julia: faster than Fortran, cleaner than Numpy".
The Coq tool/language is 32 years old and has, amazingly, only just made it to seriously discussing changing its name.
How many layers of UI consistencies are in Windows 10? Quite a lot, it turns out. moricons.dll provides quite the nostalgia trip for me..

What Does 'First Normal Form' Actually Mean? — Normalization is a fundamental concept in relational databases but there are different formal levels of it. This is a good explanation of first normal form (aka 1NF). Then you can move on to 2NF and 3NF. Fundamental database concepts worth knowing about even if you don’t memorize their definitions.

Galimatias

What Every Programmer Should Know About SSDs — SSDs have some interesting characteristics that can affect how to get the most out of them. This item led to some interesting discussion on Hacker News going into more detail about the realities of SSD use in production environments.

Viktor Leis

That Time I Deleted A Production Database, or How Everyone Makes Mistakes — A roller up Twitter thread inspired by last week’s HBO Max test email mistake.

Francis Stokes

A Tour of Bitcoin 'From Scratch' in Python — Even if you may own some Bitcoin, do you know how it really works? Maybe even the reasonably simple academic paper didn’t work for you. Here’s another take from a Pythonic perspective.

Andrej Karpathy

Is AWS Certification Worth It? Weighing All the Factors — Essential qualification or waste of time? As always, it depends.

Alex Chan

A Collection of Modern Alternatives to Classic Unix Commands — Relates to our ‘tiny tools’ section below, somewhat, but with a variety of options, each shown off with a screenshot or animation.

Ibraheem Ahmed

# Top Tiny Tools 2019-2021 🔧

supported by

htop: The Interactive Process Viewer — htop is a long standing cross platform tool that provides a better process viewing experience than standard top. v3.0 came out last year. You might also enjoy these comments by the main creator on what it’s been like to work on the project for fifteen (now sixteen) years so far.

htop maintainers

Dog: A New Command-Line DNS Client — We had nslookup, we got dig, but in 2010 we got dog too.. and it’s really neat. I’m very much here for common command line tools being ‘rethought’ and modernized in Rust.

Benjamin Sago

Ali: Generate HTTP Loads and Plot The Results in Real-Time — Go is fantastic at putting together tools like this.

Ryo Nakao

[Download] Forrester New Wave: Feature Management and Experimentation — Learn how LaunchDarkly helps development teams reduce software release headaches & enable true testing in production.

LaunchDarkly sponsor

wuzz: Interactive CLI Tool for HTTP Inspection — Think curl but with a text UI to show off request and response headers, manage params, etc.

Adam Tauber

SQLPad: A Web-Based SQL Editor to Run In Your Private Cloud — Built in JavaScript but available to run via a Docker image, this elegant SQL query runner and visualizer supports MySQL, Postgres, SQL Server, Presto, SAP HANA, Cassandra, SQLite and more.

Rick Bergfalk

nq: Simple Unix Command Line Queuing System — Small utilities for creating a lightweight job queue system in a very Unix-y way. Jobs are queued in an interesting way with chains of file locks.

Leah Neukirchen

is-website-vulnerable: Find Known Vulnerabilities in a Site's Frontend Libraries — This is a neat tool, especially as npx lets you run it right now as simply as npx is-website-vulnerable http://..... Give it a try on your own site.

Liran Tal

Falcon: An Open-Source, Cross Platform SQL Client — Built around Electron and React, this basic client can quickly do chart visualizations of query results and can connect to RedShift, MySQL, PostgreSQL, IBM DB2, Impala, MS SQL, Oracle, SQLite and more.

Plotly

duf: Think htop but for Disk Usage — For Linux, BSD and macOS and written in Go.

Christian Muehlhaeuser

Gitql: A Git Query Language and Tool — Lets you query a git repository using a SQL-like syntax, e.g. select date, message from commits where date < '2014-04-10'

Claudson Oliveira

Free Chat & Activity Feed APIs for Qualifying Teams

Stream sponsor

gping: Ping, But with A Graph — I had a shaky connection last year and wanted a more visual way to see the results of pinging 8.8.8.8 (I know, I know..) – this did the job pretty well and is written in Rust.

Tom Forbes

nb: A Command Line Note-Taking, Bookmarking, Archiving, and Knowledge Base App — Features encryption, filter/search, import/export in many doc formats, Git-based versioning, customizable color themes, extensibility via plugins, and more. Impressively entirely written in bash(!)

William Melody

And if you need encouragement to take a break..

Winddown: A VS Code Extension That Encourages You to Take a Break — After 25 minutes of use, this extension slowly fades out your syntax highlight colors, making things just annoying enough to encourage you to take a quick break.

schneefux

AWS actually discontinued an S3 feature..

Wed, 16 Jun 2021 00:00:00 +0000

#321 — June 16, 2021

Web Version

👋 I recently mentioned this newsletter is going into hiatus soon. So we're doing roundups of the best things we've linked to in the past few years, starting with a roundup of top stories/reads this week. The top half of the issue remains this week's news, as usual, but the bottom half contains links to some still enjoyable and truly golden 'oldies' :-)
__
Peter Cooper, your editor
(@peterc to follow me on Twitter)

StatusCode Weekly

What's happening in software development, ops, platforms and tools.

Practical Reed-Solomon for Programmers — Reed–Solomon codes are a group of error correcting codes used in technologies as diverse as CDs, DSL connections and RAID. Bert attempts to boil down the math to an explanation you can follow.

Bert Hubert

IN BRIEF:

The latest release of Chrome (version 91) has turned the DevTools up to 11 with a handy new memory inspector plus more performance overall by way of improvements to the V8 JavaScript engine.
It's so rare that AWS deprecates anything that S3 removing BitTorrent support is remarkable, even if it wasn't popular. Existing users will get about another 10 months of service.

Forrester New Wave: Feature Management & Experimentation — [Download Today] Learn why leading industry analyst firm Forrester has identified Feature Management and Experimentation as the latest must-have DevOps tools to enable teams to create faster release cycles while staying connected with customers.

LaunchDarkly sponsor

We've learnt that GitHub processes 2.8 billion API requests per day via its REST and GraphQL endpoints, peaking at 55K RPS. Add to that 800 million background jobs per day..
If you were online in the 90s, you might love this gallery of 88x31 GIFs that were commonly used as badges on web pages.
The Freenode IRC network, once popular with developers and open source projects in particular, appears to be pretty much 'done' after numerous incidents.
1Password has made a big push to support Let's Encrypt via matching of donations this month, if you're keen to help.
Apparently hackers managed to steal the source code to EA's popular FIFA soccer game by using Slack as a vector. It's worth checking this isn't possible at your company by looking at your chats too.

Building a WebAuthn Click Farm — How a developer has attempted to build a ‘click farm’ to ‘bypass’ Cloudflare’s proposed CAPTCHA killer with some USB security keys, an Arduino, and a bit of Python.

Luke Young

Lift 1.0: Easily Deploy Well-Designed Serverless Applications on AWS — Lift uses the AWS CDK to “expand the Serverless Framework beyond functions” and handily comes as an easily installed Serverless Framework plugin. You can then use more CDK features in your project or even ‘eject’ back out once you’re done.

lift

Privilege Escalation with polkit: Getting Root on Linux with a 7 Year Old Bug
Kevin Backhouse (GitHub)

Some Reasons Why Bugs Might Feel 'Impossible' to Resolve
Julia Evans

Disasters I've Seen in a Microservices World
João Alves

# Top Reads from 2019-2021 📖

As we mentioned recently, StatusCode is going into hiatus in July, so in a slight change to usual programming, we're doing some roundups of the most popular things we've linked to from the past couple of years. This week we're covering some of the best stories – all of which still hold some relevance (or are at least entertaining) today:

Things I Wished More Developers Knew About Databases — A then-Google-now-AWS engineer shared 17 insights about databases she’d picked up over the years. I still strongly recommend this piece and identify with lots of the points myself..

Jaana B. Dogan

The Devastating Decline of a Brilliant Young Coder — Not a technical article but an important one nonetheless. Lee Holloway essentially programmed Cloudflare into being. But then he became distant and unpredictable, and what happened to him is something that could affect any of us 😔

Sandra Upson (WIRED)

Workshop: How to Build Reusable Infrastructure as Code Patterns — Learn how to build a reusable component for a hypothetical “production ready application” in Python.

Pulumi sponsor

An Illustrated Guide to How NAT Traversal Works — Just how can peer-to-peer connections be established between two machines in spite of all the obstacles in the way? This is a thorough look at a complex problem and solution.

David Anderson

Why Senior Engineers Get Nothing Done — With a headline like this, it was perhaps no surprise this got a lot of clicks.. It's about how being a senior software engineer in an organization is not necessarily what you think it might be and may not be the actual job you want.

Swizec Teller

A Terrible, Horrible, No-Good, Very Bad Day at Slack — While you can't take pleasure in someone else’s outage, the story behind an outage can make for good reading indeed and so it went here with Slack early in 2020. It came down to how they were using HAProxy.

Laura Nolan

Why Tracing Might Replace (Almost) All Logging

Lightstep sponsor

The 'Year 2038 Problem' Is Already Causing Issues — In 2038, the number of seconds since January 1, 1970 (commonly used as an ‘epoch’ point) will exceed what can be stored in a signed 32 bit integer and this is A Problem™. Not only that, but it can be a problem now and seen in this Twitter thread.

John Feminella on Twitter

How Dropbox Migrated From Nginx to Envoy — Two Dropbox engineers took a look at Dropbox’s old Nginx-based infrastructure, its pain points, and why they migrated to Envoy, another open source service proxy.

Alexey Ivanov and Oleg Guba

How The Most Copied StackOverflow Snippet of All Time Is Flawed — An interesting part to this story was that it was the answer’s original author who also wrote the correction.

Andreas Lundblad

How io_uring and eBPF Will Revolutionize Programming in Linux — Even more exciting times are coming for development on Linux thanks to these technologies. A good overview from an engineer at ScyllaDB. This was posted over a year ago and I am already seeing exciting stuff coming down the eBPF pipeline.

Glauber Costa

CPU cores that disobey instructions

Wed, 9 Jun 2021 00:00:00 +0000

#320 — June 9, 2021

Web Version

StatusCode Weekly

What's happening in software development, ops, platforms and tools.

On 'Mercurial' Cores: CPU Cores That Misbehave — CPUs are now so advanced they are "more mercurial than precise," claims a Google engineer. At huge cloud scales, anything with even a tiny risk of corruption eventually becomes a problem! Peter Hochschild spoke on the behalf of his team at Google in a snappy nine-minute talk about the problem and shared an academic paper (PDF). Facebook has also recently written about silent data corruption at scale.

The Register

Stack Overflow Acquired for $1.8 Billion — Literally minutes after we hit send last week, this pretty big news dropped. Joel Spolsky’s blog post focuses on his thoughts and the history that led to SO being created, but apparently despite the sale, it’ll be business as usual for our foreseeable copy and pasting future.

Forbes

AWS DevSecOps Day: Bridgecrew for CloudFormation — Learn how to codify your AWS security in our hands-on DevSecOps workshop. Bridgecrew and the AWS team walk through how to automate and embed security throughout the DevOps lifecycle. Register for free with AWS credits included for a limited time.

Bridgecrew sponsor

Making JavaScript Run Fast on WebAssembly — Once upon a time, JavaScript was seen as slow but a huge amount of time, money and research went into making it super fast in the browser. Now, the Bytecode Alliance is supporting work to do the same with optimizing JavaScript performance in new environments opened up by WebAssembly (including server side).

Lin Clark (Bytecode Alliance)

Terraform 1.0 Released — The increasingly popular ‘infrastructure as code’ tool that only seems to be on the up lately. The 1.0 release is less a revolution and more a landmark release around stability and interoperability – congratulations to them.

HashiCorp

A large number of major sites went down earlier this week due to an outage at Fastly. Amazingly, the company's share price has grown by 11% since and they're better known than ever before.
Python's original creator Guido van Rossum has done a Q&A where he speculates there won't be a Python 4.0 in any reasonable span of time.
It may be no surprise that open-source maintainers are under a lot of stress, as a Tidelift survey has revealed.
In 'legal wrangling that will probably come to nothing' news, the US state of Ohio is suing Google and wants to declare that Google is a public utility and should be regulated as such.
Apple's WWDC (Worldwide Developer Conference) is taking place this week and here's a list of links to docs for the new technologies and frameworks unveiled. Or if you'd rather, you can watch the keynote.

# Tutorials, Opinions and Stories 📖

On the Diverse and Fantastical Shapes of Testing — Does your testing follow a pyramid, a honeycomb, or a trophy? How many unit tests vs integration vs other? Is mocking and stubbing OK? The testing world is certainly full of opinions and shapes.

Martin Fowler

A Look Into POKEGB: A Game Boy Emulator That Only Plays Pokémon — This is less about the code (which is less than 100 lines!) and more about how it works.

Ben Smith

12 Common Misconceptions About DynamoDB — DynamoDB’s approach is different enough to most other databases that it’s not surprising there are a variety of common misconceptions.

Rafal Wilinski

[Free Download] Guide to Effective Feature Management — Learn feature management practices to accelerate release cycles and deploy every 6 hours, instead of every 6 weeks.

LaunchDarkly sponsor

How an Engineering Team Used Python's AST to Patch Thousands of Lines of Code — This isn’t a new technique but it’s always impressive to see it working at scale. Essentially code is turned into structured ASTs, manipulated in an automated fashion, then turned back into code again.

Abdul Qadir

How Netflix Uses eBPF Flow Logs at Scale for Network Insight — Netflix has a network observability sidecar that uses eBPF to capture TCP flows in ‘near real time’ for getting insight into network usage at its large scales.

Netflix Technology Blog

Building Python Projects at Scale with Pants — Pants is a fast, scalable build system for Python apps specifically aimed at working with large repos and monorepos.

Tomas Fernandez

First Impressions of Rust Programming on Solana — Solana is a developer-focused blockchain.

Brian Anderson

On the Scaling of Uber's API gateway
Abhishek Parwal (Uber Engineering)

# Code and Tools 🔧

supported by

Links to 1211 (and Counting) Interesting Datasets — A wide variety of delights from datasets of every death in Tarantino’s movies and cruise ship inspections to the coins in 22,500 French wallets.

Data Is Plural

Amazon SQS Now Supports a High Throughput Mode for FIFO Queues — Originally SQS (Simple Queue Service) didn’t promise what order messages would arrive in, but FIFO queues, launched in 2016, changed all that at a cost of lower total throughput. Now, however, there’s a high throughput mode allowing up to 3000 messages per second per API action – a 10x increase over the standard FIFO queue quota.

Amazon Web Services

Free Chat & Activity Feed APIs for Qualifying Teams

Stream sponsor

Serverless Rules: Rules to Validate Infrastructure-as-Code Templates Against Recommended Practices — An initial preview of an AWS Labs project for extending CloudFormation and Terraform linters to provide quick serverless-specific feedback on your app templates.

Amazon Web Services - Labs

AllenNLP: An Open-Source NLP Research Library, Built on PyTorch — You’d use this for developing deep learning models to perform linguistic tasks. A new feature in this release is being able to determine fairness and bias in texts.

AI2

Althttpd: The HTTP Server That Runs SQLite.org — It’s twenty years old, it’s written in C, and you can see all the code in one file. A true example of achieving a lot with a little.

SQLite Project

Introducing the SMS Sandbox for Amazon SNS — Amazon SNS (Simple Notification Service) lets you deliver notifications via numerous methods, including SMS text messages. The SMS sandbox provides a safe environment to try the SMS features without risking your reputation.

Amazon Web Services

QUICK RELEASES:

Notepad++ v8 – Popular Windows source code editor.
Podman 3.2.0 – Tool for managing OCI containers and pods.
Grafana 8.0 – The open source observability and data viz platform.
Electron 13.0 – Build desktop apps using Node, HTML and CSS.
Vivaldi 4.0 – Alternative Web browser.