Palantir Technologies

Building Secure Search at Palantir (Verity MacKenzie, Software Engineer | London)

In this workshop you will be brainstorming and working through key problems we had designing a search system with object level authorization. Object level authorization means that we want to be able to permission which users have access on each of the documents. Palantir's secure search service is built on top of Elasticsearch (an open source distributed search system). Elasticsearch is built on top of Lucene (open source indexing and searching service) and is optimized to answer full-text searches and do aggregations (think group by queries in a relational Database). Documents in Elasticsearch map to rows in a relational database and each document has a certain mapping (think schema) and consists of a bunch of fields each of a certain datatype (like string, boolean, object with fields of primitive types etc.). Palantir has a permissioning model which contains which users (or user groups) have access to resources. In the search context each document is a resource. This workshop will cover some details of the permissioning model as well as how we ensure that the search results we return actually follow the semantic of the results being only documents that the current user is authorized to view. We will also cover challenges around performance and correctness to do this right (including race conditions around the permissioning model changing during queries).

After the workshop we will head down to Frati for some nice food and drinks.

About Palantir Technologies

Today’s critical institutions don’t just protect public safety or provide vital goods and services—they also deal with astonishing amounts of data. At Palantir, our mission is to help the world’s government agencies, Fortune 500 companies, and nonprofits transform how they understand and use that data.

We build software that lets organizations integrate and analyze their data so they can solve problems they couldn’t before. For example, we’re helping our customers combat terrorism at home and abroad, investigate and prosecute financial fraud rings, offer more credit to retail banking customers, manufacture products faster and more safely, and fight violent crime in the world’s great cities. And we’re just getting started.

If you are interested in an interview later in the semester, feel free to send your CV (in english) to Inga Penkina - ipenkina@palantir.com.