✨Introducing Peekalink, an API to magically preview any link.
Link previews is an incredibly useful feature to have in social networks and messaging apps.
They add valuable context to what otherwise would just be a string of undecipherable characters like https://youtu.be/dQw4w9WgXcQ, in other words, it turns sharing links into sharing content.
This in turn sparks users conversations and discussion, increasing your user engagement, usability and satisfaction.
It is then no surprise that link previews were one of our most requested features at our social network, Okuna.
As a community-driven platform, we set out to implementing them. Easy peasy right? Folk, were we wrong.
🙅♂️ The problem
To our surprise, doing link previews in a reliable, secure and privacy-friendly is surprisingly complicated.
Why? Because…
- Some websites have meta tags that provide a nice title, description and thumbnails, most don’t.
- Some websites have a semantically valid HTML structure to generate some reliable fallbacks, most don’t.
- Some websites have bandwidth friendly thumbnails and icons, most don’t.
- Some websites are server-side rendered and thus SEO friendly, many are not.
Apart from that, when we’re talking about previewing links, links are not just html documents, they can also be Images, GIFS, videos or even specific platform content such a Tweets or a Youtube videos that have some more metadata than your average website.
As if things were not complicated enough, the actual content of a link might also be behind many redirects, a redirect that can also change and suddenly point to something new.
Cherry on the top? Of course the actual link content can change. Its title, description, thumbnail, the amount of Youtube views, the amount of retweets. If you want to keep these previews blazing fast while also being up-to-date, things get even more complicated.
🕵️♀️ Seeking the easy-way out
At this point we were feeling quite overwhelmed. This was supposed to be another cool feature to add to our social network implemented in no more than a couple days, instead it looked like something that might as well be its own product/service.
You might be telling to yourself, well, if its such an ubiquitous feature of platforms and its not trivial to implement, surely someone has built a product/service for this right? Nope.
A quick search will show you that there’s a couple of services claiming to do so, yet all of those services implement this functionality exclusively for html documents in the most basic, unreliable and slooooow way.
What they do is just proxy your request through their servers and dome some document.querySelector on it to extract og:* metatags hoping that everything is there.
Now if you’re building an app for yourself or a tiny group of users that might be okay with things not working out 24/7, everything more than this is an overkill. However, if you like us, need to deliver a consistent, complete and blazing fast experience always, you want to do better and address all of the issues we mentioned so far.
😱 A nasty surprise
Having no options left, we set out to build the links preview service we wished we would have found, on that could be implemented in a couple hours and moved on building our product.
While doing the initial scoping of the work, we looked into how other platforms such as Slack, Discord and even Signal were doing their previews.
To our shock, we realised most of the implementations on these platforms posed security and/or privacy risks and bandwidth costs unbeknown to their users.
How they preview links varies a lot by platform, how they usually do it is by downloading and generating the the link previews in your device, this means leaking your IP to the website you’re previewing, just by seeing the link on your stream, sometimes even running Javascript code of the website in your device, posing a potential security risk and significantly draining your battery.
To make things worse, the images on the link preview thumbnails are loaded directly from the website unoptimised, resulting in a huge bandwidth drain from the application which if on a mobile phone plan, can be costly for users.
The only platform we saw trying to actively mitigate these risks was Signal, which allowed to do this by disabling the functionality entirely.
Around the same time we were putting together an article detailing our findings, security researchers Talal Haj Bakry and Tommy Mysk beat us to it and ever since the media has catched up on their research, unfortunately the same cannot be said about the platforms.
Taking this into consideration, we also built Peekalink to protect its users of some 97% of these security and privacy risks (more on this here). The 3% left unfortunately requires a bit of work from your side, better explained in our docs.
🪄 A magical end result
After countless of hours put into building Peekalink, fixing all the aforementioned problems and more, the result is almost magical.
You pass a link to Peekalink like
And you get a response like
And this does not only work with html pages, it also works with images, gifs and videos (more coming soon!).
Cool huh? Here’s a couple of things that Peekalink does in the background
- Follows redirections and the web’s caching and invalidation mechanisms gracefully
- Upgrades requests to HTTPs if available
- Uses semantic text analysis to find missing titles and descriptions derived initially from meta-tags and semantic HTML
- Downloads, optimises and stores icons and thumbnails which it then servers through a global CDN significantly reducing your users bandwidth
- Enriches platform links with platform specific details, such as Youtube views and Twitter likes through custom SDK integrations
- Following our privacy-first approach to everything we do, it analyses links connections to provide information on whether the previewed link contains trackers and will compromise your users privacy when opened
- Monitors links for changes in order to deliver up-to-date previews
- Works with Javascript rendered websites using disposable sand-boxed machines
🧙 Where to go from here?
Take Peekalink for a spin or read through our docs to dive into the technicals.
While predominant on social networks and messaging apps, links are the glue of the internet and are everywhere. We can’t wait to see what other applications can Peekalink facilitate.
— Joel
Is there something we missed you’d like to see? Let us know by pressing the chat button on the bottom right of our website!
