The problem starts when you click a link somewhere and it opens in the wrong one. Slack link? Should probably go to Edge since I’m signed into work accounts there. A YouTube link from WhatsApp? Firefox, because that’s where I’m logged into my personal Google account. Windows makes you pick one default browser and that’s that. Every single link opens in that one browser, regardless of context.

I got tired of copying links, switching browsers, and pasting them. So I built Turnstile.

How it works

Turnstile registers itself as your default browser. When you click a link anywhere on your system, instead of opening it directly, a small popup appears near your cursor showing all your installed browsers. Each one has a number next to it. Press that number, and the link opens in that browser. Press Escape and nothing happens. The whole thing takes less than a second.

It also detects browser profiles. If you have multiple Edge profiles or Firefox profiles set up, they show up as separate entries in the picker. If you have a work profile and a personal profile in Edge, you can pick which one to use right from the popup.

Automation rules

After using it for a while, I noticed that I always open Slack links in Edge, and YouTube links in Firefox. So I added automation rules. You can define a rule that says “always open links matching *.slack.com in Edge”, or “always open links from the Slack app in Firefox”. Turnstile matches the URL against your rules and skips the picker entirely if it finds a match.

Rules can match on domain patterns or regular expressions, and you can also filter by the application that opened the link. So you can say “links from Discord should always open in Firefox” and that works too.

Rule suggestions

This is my favourite feature. Turnstile watches which browser you pick for each domain and source app. After a few consistent choices, it suggests a rule. It tells you something like “You’ve opened links from slack.com in Edge 12 out of 13 times. Create a rule?” If the suggestion makes sense, you accept it and never have to pick again for that domain.

It won’t badger you with suggestions. It needs at least 5 events and 85% consistency before it even brings it up. And if you dismiss a suggestion, it won’t ask again.

Source app detection

This was a fun one to implement. When you click a link in an app, Windows passes the URL to the default browser, but doesn’t tell the browser which app sent it. Turnstile works around this by using P/Invoke to call into ntdll.dll and figure out which process actually invoked it. If that fails, it falls back to checking which window was in the foreground. This is how it can let you create rules based on which app the link came from, not just the URL.

How I built it

Here’s the part that surprised even me — I didn’t write a single line of code for this project. The entire app was built using Opencode GLM 5.1. I described what I wanted, iterated on the output, and ended up with a fully functional Windows app without writing any code myself.

Turnstile is a WPF app written in C# on .NET 9. WPF turned out to be the simplest way to get a lightweight, native-looking popup on Windows. The picker window positions itself near your cursor, animates in, and auto-closes if you click away from it.

All the configuration — rules, browser order, usage data — lives in JSON files under %LOCALAPPDATA%\Turnstile\. No database, no background service, nothing running when you’re not picking a browser.

Where to get it

You can find it on the Microsoft Store by searching for “Turnstile”. More details at turnstile.perryizgr8.com.

Most online apps and services seem to be falling into this trap. At launch, they solve an important problem or provide great value to users. People start using it en mass. As the userbase grows, the platform starts making changes that are at odds with what users want.

Some examples (including one which is a physical product):

• Apple (and subsequently every other phone manufacturer) removed the headphone jack from its phones.
• Reddit practically killed its API which allowed it to grow in the first place.
• Google search, once adorned with tasteful discreet text ads, is now often filled to the brim with ads indistinguishable from real search results.
• Facebook and Instagram have gradually increased the amount of ads to such an extent that it is common to see an entire page full of ads on your feed.

Today I want to talk about a recent change in YouTube that I haven’t seen anybody else mention. I believe this one also falls squarely in the enshittification bucket.

What is “Watch Later”?

Watch Later is a special playlist on YouTube that you can add videos to…watch later. I find it very useful because I don’t always have time to watch a 15 minute video right now, but I still want to watch it at some point. When I have time, instead of hunting for videos I missed, I just go to my Watch Later playlist and select what I want to watch.

How was it wasy to use?

The playlist was special because it was very easy (usually a single click or tap) to add a video to it.

On phone, in the feed, you could tap the three dot menu on any video, and the first option in the menu was watch later. Two easy taps, and you’re done.

On web, if you hover over the thumbnail of any video, there was a dedicated watch later button on the bottom right. One click, and you’re done.

On TV, you could hold the select button on any video and again, the first item in the menu that would open would add the video to your watch later list. One long click, and a short one.

How has it changed?

On phone, the watch later option is still there in the three dot menu, but it’s the second one. The first one is now an ad for their premium subscription. The change in positioning was very clever. It always makes me pause for a moment now, before I can tap the watch later option.

On web, the hover shortcut is gone, replaced by two useless hover icons which I will never use. Now you need to locate the three dot menu, click it, and choose the second option in the list.

On TV, they got rid of the long select menu entirely. There is no way to add a video to watch later from the feed.

Why did they make it worse?

It really feels like someone at YouTube had a grudge against the feature for some reason. It has been made harder to locate and use on every platform more or less at the same time. I have thought a lot about it but cannot figure out why they would do this. It doesn’t make any sense at all. I am sure the justification involves something something advertisers, watch time, something…

/rant

Nesting

Consider the following function that performs the common spline reticulation procedure.

int reticulate_the_splines(network, curves) {
    if(curves.size() > 0) {
        if(network.status() == ONLINE) {
            for(auto c: curves) {
                network.add_curve(c)
            }
        } else {
            std::cout << "The network is not ONLINE.";
        }
    } else {
        std::cout << "No curves to add to the network.";
    }
}

Inside the function, there are up to 3 levels of nesting. The deepest nesting is on the line:

                network.add_curve(c)

In more complicated functions, it is common to encounter even deeper levels of nesting, due to if...else conditions and loops.

Refactoring

Let’s try to decrese the level of nesting in the above function.

int reticulate_the_splines(network, curves) {
    if(curves.size() <= 0) {
        std::cout << "No curves to add to the network.";
    }
    if(network.status() != ONLINE) {
        std::cout << "The network is not ONLINE.";
    }
    for(auto c: curves) {
        network.add_curve(c)
    }
}

This function only has at most one level of nesting. I find it to be much more readable. The control flow seems to be more clearly visible. At a glance you can understand that having no curves or a network that’s not online are error states that prevent the function from carrying out its operation.

Once the error checking is out of the way, it is very clear that the function loops over curves and adds each to the network. In the nested version, this main logic was buried deep beneath condition checks, and the error handling was dispersed across the entire length of the function.

The video makes a very detailed and visual point. If you aren’t convinced by my justification here, be sure to give it a watch!

Last time we saw how to use range fields to store range values in Elasticsearch. You could store two discontinuous brightness ranges supported by a TV operating in two modes (SDR and HDR). However, we did not associate the ranges with their respective modes.

To do that properly, we need to make use of nested types in Elasticsearch.

PUT televisions_nested_index
{
    "mappings": {
        "properties": {
            "name": {
                "type": "text"
            },
            "brightness": {
                "type": "nested",
                "properties": {
                    "mode": {
                        "type": "text"
                    },
                    "value": {
                        "type": "integer_range"
                    }
                }
            }
        }
    }
}

Let’s put in the same TVs as last time, but with the mode information attached.

PUT televisions_nested_index/_doc/sony_bravia
{
    "name": "Sony Bravia",
    "brightness": [
        {
            "mode": "SDR",
            "value": {
                "gte": 100,
                "lte": 300
            }
        }
    ]
}

PUT televisions_nested_index/_doc/lg_oled
{
    "name": "LG OLED",
    "brightness": [
        {
            "mode": "SDR",
            "value": {
                "gte": 100,
                "lte": 400
            }
        },
        {
            "mode": "HDR",
            "value": {
                "gte": 500,
                "lte": 1000
            }
        }
    ]
}

Now we can query for a specific brightness range in a particular mode. So let’s search for a TV that supports 600 to 700 nits in the HDR mode.

GET televisions_nested_index/_search
{
    "query": {
        "bool": {
            "must": [
                "nested": {
                    "inner_hits": {},
                    "path": "brightness",
                    "query": {
                        "bool": {
                            "must": [
                                {
                                    "term": {
                                        "brightness.mode": {
                                            "value": "HDR"
                                        }
                                    }
                                },
                                {
                                    "range": {
                                        "brightness.value": {
                                            "gte": 600,
                                            "lte": 700
                                        }
                                    }
                                }
                            ]
                        }
                    }
                }
            ]
        }
    }
}

This will return the LG TV since it is the only one that supports our required brightness in the HDR mode.

Thanks to the person who answered my question on Stack Overflow. It helped me make progress when I was completely stuck.

Elasticsearch is a great index and document based search system. It’s amazing for storing a bunch of data and retrieving the right bit very quickly. It has a rich querying system that is highly flexible and overall it’s a drop-in, batteries-included search engine that you can host.

I recently found out that Elasticsearch supports range field types. Instead of assigning a single value to a field, you can instead put in a range. These are called range field types. You can then make ranged queries to look up these kind of fields.

Suppose you want to store the attributes of TVs in your index. One of those is the brightness of the TV which is commonly measured in nits. Let’s create an index with a range field that can store this info.

PUT televisions_index
{
    "mappings": {
        "properties": {
            "name": {
                "type": "text"
            },
            "brightness": {
                "type": "integer_range"
            }
        }
    }
}

Now we have an index ready to accept documents about each TV we want to store. Let’s store our first TV. This is a Sony Bravia TV whose screen brightness can vary from 100 nits to 300 nits.

PUT televisions_index/_doc/sony_bravia
{
    "name": "Sony Bravia",
    "brightness": [
        {
            "gte": 100,
            "lte": 300
        }
    ]
}

Here gte and lte stand for “greater than or equal to” and “less than or equal to”. This signifies a range of 100 to 300 inclusive.

Now let’s store another TV. This LG OLED has two modes. In the standard mode, its screen can go from 100 to 400 nits. But in the HDR mode, the screen can go from 500 to 1000 nits.

PUT televisions_index/_doc/lg_oled
{
    "name": "LG OLED",
    "brightness": [
        {
            "gte": 100,
            "lte": 400
        },
        {
            "gte": 500,
            "lte": 1000
        }
    ]
}

Now let’s make a simple query to our index. We want to get a list of TVs that support a brightness value of 700 nits.

GET televisions_index/_search
{
    "query": {
        "term": {
            "brightness": {
                "value": 700
            }
        }
    }
}

This will return only the LG OLED TV in the result, since only that one supports a brightness value of 700 nits.

We can also query for a range of brightness values.

GET televisions_index/_search
{
    "query": {
        "range": {
            "brightness": {
                "gte": 150,
                "lte": 250
            }
        }
    }
}

This will return both TVs in the result, since both of them support 150 to 250 nits of screen brightness.

Google’s bait and switch

Google offered their excellent Google Photos app and backup service for many years, with unlimited photo/video backup. The only catch being that your media is compressed and uploaded. But the resulting quality was good enough that I did not mind. My wife and I used it for backing up all our photos, and sharing them with each other.

They ended the unlimited free tier in June 2021. Now we must pay a monthly rent to Google to be able to keep using it as our backup.

Google’s nonsensical pricing

I know that my photo library is over 200 GB, maybe around the 1 TB mark. But it is continuously growing. I am sure it will cross 2 TB in the next 5 years, given the ever-increasing quality of our cameras.

So paying every month for 2 TB now, and then inevitably being forced to pay more and more indefinitely is not a very enticing thought. In my opinion, they should charge a low fixed amount (say ₹50) per month, regardless of the amount you’re storing, plus a very low amount per gigabyte (say ₹0.0002). That way, everyone pays for the base service equally, and everyone also pays for the storage they are actually consuming.

PhotoPrism

The first alternative I found was PhotoPrism, a self-hosted app that you can run on your server, and access from your phone or web. Point it to a folder on your drive and it imports all photos and videos.

Pros:

• Easy to set up.
• Runs on docker, so platform-independent.
• Open source and free.
• Feature-rich, includes face recognition and a powerful search.
• Actively developed, few bugs.

Cons:

• No good way to backup from phone to server.
• No native apps on mobile, have to use horrible and clunky PWA.

The last one is extremely annoying. PWAs are simply not ready. It literally does not feel like a proper app. It feels like a web page. Each button press is laggy, every photo takes a while to load, all around a pain to use. Infuriatingly, it seems like there are no plans for a native app at all. Apparently, the PWA offers an almost native app-like experience. No, PhotoPrism, it doesn’t.

There is an unofficial Flutter app, but it is currently incompatible with the latest version of PhotoPrism. As far as I can tell, it has been incompatible for over 2 years. So I don’t have high hopes from this project.

Immich

Recently, I stumbled upon Immich, a self-hosted app that has the explicit aim of replacing Google Photos. To that end, it has native mobile apps that work very similar to Google Photos. The apps seem highly polished, and the server is easy to set up using docker compose.

Pros:

• Easy to set up.
• Runs on docker, so platform-independent.
• Open source and free.
• Slick native Android and iOS apps.

Cons:

• Not very feature-rich yet.
• Very active development means things are likely to break.
• Can’t import existing photo library.

A strange quirk with Immich is that you can’t point it to your existing photo library and have it scan it (as far as I can tell). You must upload photos from your phone using the backup feature, or upload pictures via the web app. In my opinion they went too far in replicating Google Photos here.

Problem with Immich

I set up the server on my PC, downloaded the app on my phone and started the backup process. The backup always gets stuck on a particular video file. It’s not overly huge, about 150 MB. No matter what I did, it always gets stuck on this file and the backup never completes.

How my server connects to the internet

My “server” is just my PC in my bedroom. It runs Windows 11. I have set up a Cloudflare tunnel to expose a port to a particular DNS record. I input that DNS name in the Immich app’s server setting. I need to do this because I don’t have a public IP from my ISP, and all ports are effectively blocked due to a double NAT.

Debugging

I had a suspicion that the issue had something to do with the tunnel. Since the upload gets stuck on a slightly large video, and not on a regular-sized photo, I wondered if the tunnel was restricting uploads above a certain limit. After all, Cloudflare’s architecture is built to support small files, and that too in the other direction.

I started a python server using the uploadserver package. It lets you download and upload files via a rudimentary web interface. I tried to upload a video file of about the same size:

~> curl -v 'https://mysite/upload' \
  -H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary1dMEK7Ts8bf8AZTg' \
  -F "files=@/Users/parikshit/Downloads/clip.mp4"
*   Trying xxx.xxx.xxx.xxx:443...
* Connected to mysite (xxx.xxx.xxx.xxx) port 443 (#0)

...skipped...

< HTTP/2 413
< date: Mon, 20 Mar 2023 14:29:51 GMT
< content-type: text/html
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=******"}],"group":"cf-nel","max_age":604800}
< nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< server: cloudflare
< cf-ray: *****-BLR
< alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400
* HTTP error before end of send, stop sending
<
<html>
<head><title>413 Request Entity Too Large</title></head>
<body>
<center><h1>413 Request Entity Too Large</h1></center>
<hr><center>cloudflare</center>
</body>
</html>

As suspected, Cloudflare doesn’t like it if you try sending large files in the opposite direction. Possibly their network is simply not optimised for that.

Solution

There are two possible solutions I can think of.

1. Immich is open-source. I could contribute a patch that chunks larger files when uploading.
2. I could use a Wireguard or Tailscale tunnel between my PC and some cloud server, and use that to route my traffic.

I’ll post an update when I manage to solve the issue.

Update

I submitted a PR to Immich’s GitHub repo that simply adds the header Transfer-Encoding: chunked to the request when uploading a file from the Flutter mobile app to the server. This makes the HttpClient omit the Content-Length header, which bypasses CloudFlare’s checks, and our file upload completes successfully.

The PR was merged and my changes are live in version 1.52.0 of the app.

As is true for many software teams today, ours is a mixture of people from multiple cultures and backgrounds. Along with all the bonuses this brings, there is also the small inconvenience of varying accents.

I consider my accent-understanding skills to be decent. I can make sense of all but the most extreme ones. And I like to think mine is at least intelligible to most others.

Face-to-face conversations in person are rare, since our team is highly distributed across geographies and timezones. We use Google Meet and Slack huddles to converse with each other. Both of these tools’ audio quality is excellent (although Google Meet’s screen sharing quality leaves a lot to be desired).

Everybody’s voice comes out crystal clear, even when network connections are not the best.

In recent times, I’ve started noticing something. Some people’s voice seems to be extremely low fidelity. It sounds like they are on the other side of a kiddie walkie-talkie, operating at the limit of its range. Coupled with my inability to comprehend all accents equally, it gets really hard to follow what’s being said.

I was puzzled, because the same meeting would have everybody loud and clear, and a couple of people muffled and quiet and generally low-quality. Over the last few months I’ve figured it out.

Everybody who sounds muffled is using an Apple airpod. That’s it. The mic on this fancy device is worse than on children’s toys from the 90s.

So whoever is reading this, if you’re using an airpod, please don’t use it in meetings. It will help you get your point across much more clearly if you ditch the low quality mics that are built into the contraption.

Thanks for tuning into my Apple rant of the week.

I saw this tweet making the rounds last week. If you don’t want to visit Twitter I’ll describe what it says.

Harsh Goenka says:

I enjoy #SharkTankIndia as a program and I think it is a great platform for our budding entrepreneurs.

But whenever I think of sharks, I think of the movie ‘Jaws’ and bleeding 🩸!

There’s an image of a table like so:

The point is that all the sharks (with the exception of Aman Gupta’s boAt) are presiding over failing, unprofitable companies.

Thoughts on the first season

I quite enjoyed the first season last year. I had previously watched the UK show Dragon’s Den which seems to have inspired the string of Shark Tank productions around the world. I liked the Indian take on the subject matter and was honestly surprised at the variety of ideas and determination with which people are trying to realise them across the country.

What’s changed with season two

The show lost its most controversial shark, Ashneer Grover. Though they haven’t come out and said it, his disgraceful exit from his own company must have played a role in the decision to replace him.

Apart from this, I feel the show has changed its tone significantly. Sharks seem to be much more reluctant to partake in risky bets. They fail to ask pertinent questions, and seem to be eager only when presented with a perfect company which has almost no risk at all.

The core issue - lack of focus on what’s important

In the British Dragon’s Den, a very important question that every entrepreneur got asked was “What are you going to do with the money, if you were to get it today?” When someone is looking for an outside investment, they must have some plan in mind that they think will take the business to the next level, but they just don’t have enough cash to put the plan into action. Hence, the appeal to investors.

So if you can’t explain what you plan to do with the money infusion, it used to be a sure-shot failure to secure funding in Dragon’s Den.

This question is totally missing from the second season of Shark Tank. Not one shark bothered to make this all-important query of any pitcher.

Oh you want 50 lakhs? What is your sales last year? What is your profit margin? What is your projection for the next year? What are your qualifications?

Ahh, that’s a nicely running profitable growing business! I’d like me a piece of it. Here’s your money!

If someone has a business that’s already doing well, and they don’t have any plans to make it explode using a capital infusion, why would they ever come to you? They’re already running a profitable venture. They can continue doing whatever they are and collect their profits.

Another glaring factor that’s missing in the second season is the broke inventor who has a brilliant idea, has done basic market research and needs seed capital to get his company off the ground. Sharks in season two will laugh such a guy out of the show, telling him to come back when he has a business. That’s why he came to you for help! If he is able to prove his idea in the market and start making good money off it, why would he come back to you?

Not supposed to be a show about investment

What the sharks don’t seem to understand is that the point of this show was never to be an investment show. It was supposed to promote the spirit of entrepreneurship. It wanted to ensure that no good idea goes un-funded. If you have an idea and you can convince these industry stalwarts of its worth, you don’t need to worry about your first capital investment to get your venture started.

Intriguingly, the first season was much better in this regard. There were multiple lone inventors who appeared on the show and a few of them were also able to secure funds.

Is it because of the sharks not being profitable?

Now, we come back to the tweet. Are these problems in the season because the sharks are simply not good enough businessmen? Are they just incompetent as entrepreneurs? After all, their companies are bleeding money by the hundreds of crores. What do they know about running a profitable business?

I don’t know if that’s what the issue is, but it’s definitely something to think about.

As the app grows, especially if it is written in a “slow” language like Python, you will find yourself trying to jump for a micro-service for every new feature that comes your way. This is because it becomes much harder to maintain quality and performance in a Python/Django monolith as the codebase grows.

The general tendency in this case is to copy the existing Django project, make your new addition, remove or comment the existing unneeded stuff and deploy it somewhere else. But cruft accumulates, every bad decision made in the app carries forward.

The ideal thing to do in such situations is to create a new service from scratch, preferably in a faster language like Go or Rust. However it’s hard to do, and takes a lot of time and effort. This additional delay is hard to justify for a small to medium feature.

In times like these, it is helpful if you have a ready-to-go template to start you off with basic scaffolding already in place. You can start from “Hello, world!” within a minute. Then add your required functionality.

Enter Cookiecutter.

It’s easy to create a template using Cookiecutter for a basic Go app that serves a REST API using Echo. It’s set up with common amenities and has a nice starting point for any service.

Naive solution - file checksum

A super easy solution is to store a list of checksums for your files in a spreadsheet, like so:

Whenever you are about to add a file to your collection, calculate its MD5 checksum ($> md5sum evil_corp.pdf) and search for the checksum in your spreadsheet. If you don’t find it, add it as a new row, confident in the fact that you’re not duplicating anything.

Why this fails

Any small change to the document will cause the checksum to change completely (Avalanche effect), even if the change is insignificant or even invisible to the reader. PDF files have metadata in them, stuff like /Producer, /CreationDate, /ModDate, etc. For my purpose, I don’t care about these things. I want to know if the content of the PDF is same or not.

Also, some PDFs are generated on the fly when you request for them, and the generator program will put a small Generated on 29-11-2022 somewhere in the document, probably in the footer. Again, even though this is technically part of the PDF content, and is also visible on the document, I don’t care about this.

Smarter solution - image checksum

Open the PDF file with a utility like PyMuPDF. This gives you an option to render the document to an image. Then you can checksum the generated image instead. To handle files with multiple pages you can checksum each page’s image, concatenate the checksums, and then checksum this concatenated string.

Why this fails

This solves the metadata problem, you are now checksumming the visible part of the document, which is what you care about. Metadata can be totally different, but that won’t affect your deduplication. It still doesn’t do anything about the second problem. You don’t want to strictly consider every pixel in the document. I want to tolerate small differences (a different date in a footnote, an extra box drawn around a word, etc.), and still consider such files identical.

Locality sensitive hashing

My search for a suitable solution led me to this wikipedia page.

locality-sensitive hashing (LSH) is an algorithmic technique that hashes similar input items into the same “buckets” with high probability. Since similar items end up in the same buckets, this technique can be used for data clustering and nearest neighbor search. It differs from conventional hashing techniques in that hash collisions are maximized, not minimized. Alternatively, the technique can be seen as a way to reduce the dimensionality of high-dimensional data; high-dimensional input items can be reduced to low-dimensional versions while preserving relative distances between items.

Pay attention to the last part:

high-dimensional input items can be reduced to low-dimensional versions while preserving relative distances between items.

Sounds like exactly what I want. I want to reduce a PDF to a low-dimensional checksum, which remains comparable between different PDFs.

This is quite new to me, so I am going to read all I can about it before I try to apply the technique to my problem. I am listing some links that I plan to examine.

• ImageHash on PyPi
• Description of average hash and perceptual hash
• Description of difference hash