Category: Technical

De-crufting Google Search

Just a quick one, but I know some folks might want to know this: it sounds like Google finally added a way to remove all the AI and other self-insert bullshit from their search results. This post by Ernie Smith over on Tedium explains more, but the TLDR is: at the end of your search query URL, add udm=14. So, for example, if you do a search for Buckaroo Banzai, then go into the URL of the search and update it to https://www.google.com/search?q=buckaroo+banzai&udm=14, all the cruft drops away and you just get links to stuff again. (That may seem like kind of a pain, but as that post points out, there are ways to make your searches insert that automagically, depending on your browser.)

I’ve mostly migrated away from Google search and tend to default to DuckDuckGo, but it’s still good to know. (Personally, I’d love to see a way to customize which tools get turned on or off. Like, I don’t want the AI, and I don’t need the quora answers etc, but it’d be nice to leave the calculator feature turned on. Stuff like that.)

The Soul of the Web

Over at The Atlantic, Kaitlyn Tiffany has an article on The Battle for the Soul of the Web – a headline that may sound a little dramatic, but is touching on some important topics. The article touches on a few different topics, but a central one is about the decentralized web (DWeb), and where that both intersects and contrasts with the Web3 space. It’s an interesting topic that is worth a deeper dive if you care about online spaces at all.

Decades removed from the gonzo highs of blinging GIFs and wacky blogs, the web is now a place where many people feel exploited, manipulated, and tracked; where freedom of speech is being tugged around in a strange culture war; and where the rich get richer.

Among this set, one solution seems to be the consensus favorite. If these problems are intrinsically linked to consolidated tech giants like Meta, Google, and Amazon, why not embrace technologies that decentralize power? 

Kaitlyn Tiffany

I’ve been railing against and ranting about information silos for years, so I suppose it’s no surprise that this line of thought might be a little appealing to me. I’ve had a passing interest in the potential of technologies like IPFS for years, for instance, though it remains to be seen how things pan out, and what adoption looks like – how much broad adoption is necessary for a technology or philosophy on technology to sustain itself and become independently viable?

That said, I think they’re at least asking the questions, and that’s a good start:

Nathan Schneider, a media-studies professor at the University of Colorado at Boulder and a frequent writer on collectivism and tech, told me. “DWeb asks more,” he said, and dwells on two key questions: “What do we actually want socially, and how do we center those values in our technical designs, so the technical becomes a means to an end, rather than an end in itself?”

There’s obviously some overlap between some decentralized web efforts, and decentralized finance efforts. I hope that the former doesn’t get dragged down because of the latter (or, more specifically, the scam-filled gold-rush that congealed around DeFi in recent times). There’s lots of questions I don’t have answers to about this (perhaps others do?), like how do we make the technology useful, approachable, ubiquitous… without also making it a target for those who want to exert power and control, and exploit it? User congregation around particularly useful or interesting tools or information is inevitable, and with that, how do you prevent just making silos all over again? (Another topic that gives me a headache is how to shepherd information accuracy, and curb disinformation in a world where oversight is virtually impossible?)

Site Monitoring, Plausibly

I’m on my annual sojourn to Squam this week, listening to the lake and the loons. I’ve got a few personal projects I’ve been meaning to do that I hope to work on while here (in addition to the socializing and swimming and such). For instance, I’ve been meaning to move away from Google Analytics for monitoring site traffic. No diss on the quality of the tool, but philosophically I’ve been increasingly uncomfortable with handing over my site’s viewership information to Google for their own purposes. I do still want some level of monitoring, though: site stats aren’t strictly necessary, but I’m enough of a stats geek that I do like to occasionally see what pages are getting views, what links are getting clicked, and so on.

There’s a couple different open source solutions out there that serve this purpose. Matomo is a popular one (it was previously named Piwik and has been around for quite some time), but seemed excessive for what I wanted. Personally, I ended up landing on Plausible. It’s fairly lightweight, fast, GDPR compliant, doesn’t collect PII, and I can self-host it. Setup was a little bit of a pain, but not an insurmountable one. To be more clear: installation itself is easy, as it’s just a docker container, but it terminates to http, not https. This meant I needed to set up a reverse proxy to pass it through a secure connection, or else browser content protections would block the script (they don’t like mixing secure and non-secure files). But hey, good opportunity to learn something new!

Link: Aqua Screenshot Library

This is for the UX and UI nerds out there: the Aqua Screenshot Library. It’s a collection of various windows, dialogs, screens, and other UI elements in Apple’s Aqua interface in each major version since the Mac OS X public beta. I’ll be curious to see how they handle cataloging Mojave, since there’s light and dark modes for everything. It’s really interesting to see how the interface has continued to evolve (and which aspects remain largely unchanged). You can really see where and how this:

Became this:

Certbot Gotchas

As a project this week, I’ve been doing some backend maintenance for my web hosting, which includes getting everything set up with SSL certs through Let’s Encrypt. (The writing is on the wall: most sites that can switch to HTTPS should switch to HTTPS. Not just for the added security for you and your viewers, but also because browsers and search engines and similar are starting to give warnings if it’s NOT secure.)

Thankfully, the days of having to pay an arm and a leg for a cert have passed. Let’s Encrypt is a free non-profit service (in partnership with other orgs including EFF, Mozilla, and Akamai, to name a few), which generates free, short term SSL certificates for your site. (For larger organizations, you may want to still throw down on a longer term set of certs, but for personal use this is great.)

Using the certs is pretty straightforward: they’ve created a tool that can run on your web server called certbot which streamlines the process and also monitors and automatically renews the certificates when they’re close to expiration. Installing certbot is pretty straightforward: it’s available via various package managers (apt and similar), so chances are good that whatever OS your server is running can install it pretty easily.

That said, there are still a few gotchas that I felt like got glossed over in the docs I was reading/following when using the tool, so here’s a few notes:

  • Be explicit about your domains: certbot doesn’t currently support wildcards (i.e. being able to say *.nadreck.me and have that handle any subdomains like images.nadreck.me). Instead, list them all out if you want them to share a certificate, and that includes both with and without www. So, for example, you might want to do something like sudo certbot --apache -d nadreck.me -d www.nadreck.me. If you don’t include both, someone going to your site using the address that wasn’t specified may end up getting the site blocked and the user warned of something potentially fraudulent.
  • If you already generated a certificate for a domain and need to update it (maybe you added a subdomain, or forgot to add www), the command you’re looking for is --expand. (I would have thought “update”, but no.) Note that when expanding a certificate, you need to re-list all domains you want included (you don’t just list the one you’re adding). So, using nadreck.me as an example again, if I wanted to add “images.nadreck.me” to the existing cert, I’d do sudo certbot --expand -d nadreck.me -d www.nadreck.me -d images.nadreck.me.
  • Keep it separated: the certs are free, there’s no need to overload the cert with a ton of domains. While it makes a certain amount of sense to bundle a domain and subdomains together, there’s no need to make one cert for all your sites. criticalgames.com shares a cert with nadreck.criticalgames.com, but not with nabilmaynard.com, if that makes any sense.
  • You can’t preemptively add sites to a cert. Certbot/letsencrypt performs a challenge response as part of the process to make sure you actually own the site you’re trying to set up, so if you haven’t actually set up that site or subdomain, the challenge will fail and the cert won’t be generated correctly. If you wanted to add, say, files.nadreck.me to your certificate, you’d need to set up that subdomain first, then expand your certificate. (The site can be empty, but the URL needs to resolve and land somewhere real.)

Anyway, hope that helps! The process really is pretty straightforward, and I recommend getting things set up to anyone maintaining a website these days.

Poking around with Python

With my usual caveat that I’m not primarily a programmer (and thus I’m sure there are better ways to do things), I thought I’d share a quick python script I put together.

For some context: I’ve been a long-time subscriber to the KEXP Song of the Day podcast, and I’m not prone to deleting songs after listening to them, which means that podcast is up to ~3000 songs, all sitting in a single folder. My car stereo has usb support, so ostensibly I can dump the whole folder onto a usb key and play it all while driving. Unfortunately, the car stereo can’t handle a folder with that many songs, so I needed to split it into smaller subfolders of around 100 songs each.

Now, I could do this manually (and did, the first time around), but I wanted an excuse to experiment with some scripting, and also wanted to fix some of the song titles (sometimes the ID3 title tag would include the artist, other times not). Doing some searching around, it seems like there’s a pretty solid python library for reading and manipulating ID3 tags, so I decided to use that.

You can see the resulting script here. It does two things: copy the files into subfolders (in a somewhat arbitrary order, which I may try to fix later), and then compare the ID3 artist tag to the ID3 title tag, and if the title tag starts with the same thing as the artist tag, strip the artist name out of the title tag (since it’s redundant, and meant the title was often cut off from being seen on the stereo’s display).

If it’s useful for you, cool! If not, I figured I’d at least share what I’ve been puttering with lately.

What I Use: RSS

I encourage people to subscribe to my site via RSS, when mentioning I have a site on the Facebooks and Twitters and similar. This may seem a little archaic (that is so 2008), but honestly RSS is still one of my go-to solutions for finding worthwhile things to read, watch, or experience.

One of the big reasons you don’t really see RSS mentioned anymore (despite folks actually using it often, without realizing it… looking at you, podcasts) is because Google stupidly shut down Google Reader, which was the de facto standard for reading your feeds. That killed a lot of momentum for its use.

While RSS may be limping along, it’s not dead, and a lot of sites actually do have RSS feeds, still — they just aren’t as prominently noted or advertised or linked anywhere.

Of course, even if you do decide to use RSS, there’s still the hurdle of finding an RSS reader you actually like. A lot of folks go with a web-based option (ala Google Reader), so they can read on whatever device they happen to be on. There’s also some pretty nice apps for sale (for instance, NetNewsWire), if you’re so inclined, and a lot of RSS-adjacent apps (like several web browsers, and even Apple Mail) are available as well. Personally, I use Vienna RSS, which is an open source project made for macOS. I’ve tried a bunch of other apps and methods, and this is the one I keep coming back to (there was a gap where development wasn’t really happening much, so I looked around a fair bit, but regular updates are happening again). It’s fairly fast, robust, and seems to handle a ton of feeds well. If you’re looking for a reader, I’d say it’s worth a try.

I recently went through and cleaned up my RSS feeds, getting rid of dead feeds. I just want to say, to all those bloggers who have continued to post after the blogging fad wore off: I salute you, and I’m still reading.

Extracting a Nested JSON Value in Python

First, some context: I’ve been working on some Python libraries at work that do things with sets of json data. This is generally pretty easy: Python has a nice library for reading json, so it can be worked on as a native dictionary object in Python. This is great for simple json objects, but there’s some pretty complex json data sources out there, whether it’s being returned as part of an API, or is stored in a file. Sometimes you need to access a specific value from a key buried a dozen layers deep, and maybe some of those layers are actually arrays of nested json objects inside them.

While both arrays and dictionaries are native to Python, so you can do this, it’s kind of a pain. Thankfully, many smart people have already been tackling things like this, which is how there are now handy libraries implementing a (pseudo-standard) approach for getting the value given a specific json path. (There’s even an online evaluator to help you craft a good jsonpath.) The one I’ve been using is called jsonpath-rw. When I looked at the docs for jsonpath-rw, I was a little frustrated, and I wanted to take a minute to write out my eventual understanding, in case there are other folks in a similar boat to me.

Note: I’m not primarily a programmer (currently I’m a Technical Writer, and prior to that I was a QA Engineer — mostly exploratory, not automation), so while my example works, there’s probably more robust and pythonic ways to do the same thing. Continue reading “Extracting a Nested JSON Value in Python”

My Take on the WWDC Keynote: Net Positive

First, if you’ve not watched the WWDC Keynote yet, you can watch it here: WWDC 2016 Keynote (You can also see a write-up over at Wired and a Liveblog of the event at Engadget.) There are a few things that came up that I think are pretty notable.

First, Continuity continues to be a big push: Apple wants as seamless an ecosystem as possible across all devices and platforms in their stable of products. We started seeing some elements of this in Yosemite and El Capitan, and it looks like they’re doubling down on it in Sierra. I have some reservations about this — mainly, lock-in and whether or not it will play well with third parties. The concept itself, though, makes a ton of sense. I’m curious to see what sort of response we’ll see from Microsoft and Google in this space (MS is starting to point this direction with their push towards a single core across platforms, but at the same time we’re seeing a de-emphasis of Windows Phone, so curious how this will play out).

Second, Machine Learning. All the big players are getting into it (Siri, Alexa, Cortana, Google Assistant), and Apple has clearly invested heavily in this area, with tight integration of Siri into iOS and macOS. One thing I think is notable about Apple’s choices with this, though, is keeping the AI on-device, rather than web-driven. I’m very curious to see how this evolves in future releases.

Third, Security, Privacy, and Encryption. Several times in the keynote, they made a point of calling out that they’re NOT building profiles of user, and are keeping PII on your device, not on their servers. This emphasis on privacy (and security) pervades a number of the choices they’re making, which I applaud them for committing to. While I disagree with some of their product decisions (single-port computers, charging ports on the bottom of the new magic mouse, etc), I genuinely appreciate that they’re sticking to their guns in the face of pressure from the government.

Fourth, Opening up new APIs. A big concern I’ve had in recent releases from Apple is continued lockdown of services, where it felt like if you weren’t Apple, you couldn’t play on the playground. This release sees several integrated services get opened up to third parties (Messages, Maps, Siri being the big three to me), which gives me some hope that Apple isn’t entirely forgetting what made OS X so great.

Fifth, Swift Playground. It’s worth noting that this closed out the keynote, and for good reason. Apple is committing to bringing programming into education in a big way, by making what appears to be a robust learning app that targets youth where they are (mobile devices like iPads), teaching them a language they can directly use for real, complex applications. This is a big win for both Apple and STEM: For Apple, it gets a new generation of developers started using their tools, environment, and language, which you can bet will make an impact on what they choose to use in the future. For STEM, they’re providing free tools, free resources (entire books, including guides on how to teach it and incorporate into your curriculum), already targeted to youth. That’s awesome. You can read more about the whole initiative at their Everyone Can Code page.

It wasn’t covered in the keynote, but has been brought up elsewhere: they’re also releasing a new Apple File System, replacing the old and creaky HFS+. This is significant: Apple’s been using HFS (and then an expanded HFS+) for basically the entire time Mac OS has existed. From reports, it sounds like a robust, next-generation file system that brings some brilliant and essential features. While we likely won’t see the OS truly make the most of these new features until the version after Sierra, this is still quite interesting, and I’m excited to see what gets done with it.

Overall, it felt like a productive developer-centric keynote. It leaves me feeling cautiously hopeful about the future of the ecosystem, and that they’re placing their bets in the right places.

Mood Driven Development

Chris over at CSS Tricks, telling it like it is:

But what is highly prized in our industry is productivity, in whatever form it takes.

“Hey, I refactored some of our mixins to be more efficient and made sure they are used properly site-wide.”

“Good morning, I looked over a lot of the copy around the site and have some ideas on what we can change to make it more clear and cohesive.”

“This afternoon I closed out a couple of long-standing bugs that have been bothering me.”

Any place I’ve ever worked, any of these things would have been applauded. Especially if they relate to the current team/project at hand. That’s what productivity is.

The danger is that you fight against urges to work on something different. You feel like you should be working on converting some layouts, and you feel guilty for tweaking some color palettes. You’re kind of into cleaning your inbox out right now, but feel like you are being lazy for not getting the JavaScript scaffolded out for that new thing. You’re finding a funny image to respond to a playful customer with, but you’re a little mad at yourself for not updating those docs.

That’s too bad, since you are being productive anyway. You’re following your mood.

Chris Coyier, Mood Driven Development