About Periplus

Another idea.
Another data pipeline.

Periplus grew out of years of small data and machine learning projects. Each began with a suspicion that some corner of the internet might contain an interesting signal.

Before the idea could be tested came the crawler, the parsers and the extraction jobs. Sometimes the resulting data revealed a better question. Sometimes it revealed a missing field or a mistake in the parser. Sometimes the idea simply wasn’t worth pursuing.

The next project brought another pipeline. A website changed its layout, and an existing parser needed fixing. Too much of the work had to be repeated before the interesting part could begin.

Periplus is an attempt to keep more of that work useful, across projects and between people.

Leave room to
change your mind.

An early extraction is a decision about what matters. It is easy to discard something that turns out to be useful later.

Periplus preserves the captured HTML and makes its text, structure and links available to query. You can return to the page as it was captured, examine its context and extract different fields as your understanding develops.

The source remains separate from the interpretation. A new question can use the same material without inheriting every assumption of the first project.

You don’t need the final
question to get started.

A directory, a collection of company websites, an unfamiliar set of publications: sometimes a source looks interesting before there is a clear project in mind.

SQL gives you a way to explore it. Search the text, pull out headings, follow links between pages and compare their destinations. Inspect the section around a link to understand its context. Change the query and see what else emerges.

When a question becomes worth returning to, repeated captures give you dated observations to compare. The same query can help investigate how a corner of the web is changing.

Start in the SQL console. Sign in to the app to write queries, run them and export results.

For people who think:
“I wonder what’s in there.”

Periplus is for curious data people: researchers, developers, analysts and hobbyists who enjoy finding a source, looking for patterns and seeing whether an idea holds up.

The collection is shaped by the people using it. Bring websites you want to investigate, and their captured pages become available to others too. Someone else may find a signal you weren’t looking for, or use your starting point for an entirely different project.

Over time, the ambition is to make interesting queries and datasets easier to share as well, so people can build on each other’s investigations alongside the source material.

Bring a question, a hunch, or a corner of the web you think is worth a look.

Explore the platform.
Understand how data is used.

The documentation is public. Discovery, retention, freshness and SQL live in the signed-in app. The core documentation covers collection, SQL extraction, and query-service limits.

Questions, queries, and privacy

Periplus logs executed SQL, parameters, outcomes, and timings to debug and improve the service.

Keep secrets and private information out of queries and parameters. Submitted collection URLs and collected data are public.

Using collected content

The corpus is shared; product access requires authentication. Periplus has not published a blanket reuse licence for third-party content or formal service terms. Do not interpret the availability of a row or an export as permission to redistribute the underlying content. Contact us about your intended use before depending on Periplus for redistribution or production.

Formal privacy and data-use policies are not published yet. For questions about stored information, source content, or removal requests, contact [email protected].

Help shape Periplus

Tell us what you were trying to find, which sources you needed, and where the results helped or fell short. Include the query reference when reporting a problem so we can investigate the same execution.