What are Pandio Service Accounts?

Pandio is fortunate enough to run some of the largest Apache Pulsar installations in the world for our customers. In doing so, we identified a need to provide additional functionality to the administration of an Apache Pulsar installation.

At its core, Service Accounts are a wrapper around the administration tools of Apache Pulsar. This wrapper provides helper methods to make administration of a cluster(s) easier, provide further granularity by allowing a super user command to be executed by an administrator rather than granting the full super user role, leveraging URNs (user resource name) to make it easier to understand what is happening in a cluster(s) without having to perform ad hoc lookup requests, and much more.

Here are the major things that Pandio experienced while managing Apache Pulsar installations that Service Accounts address:

1. Management of thousands of tokens on tens of thousands of topics becomes difficult.
2. Management of permissions across clusters is time consuming.
3. Allow some super-user functionality without granting a full superuser.
4. Instantly revoke access to multiple clusters and thousands of topics.
5. Gain easier observability into who is doing what inside a cluster in real-time.

Millions of Topics

One of the amazing things about Apache Pulsar is that it can handle millions of topics. However the ramification of this is the inevitable difficulty in managing access to this many topics. Service Accounts solve this by abstracting the hundreds or thousands of API calls that are needed to achieve a single business goal of authorizing a user or revoking access for a user. A quarterly task to audit access to Apache Pulsar could take weeks, now takes hours with Service Accounts.

Better Security

A user token was compromised. What now? Before Service Accounts, that would require hundreds to thousands of API calls to discover the associated resources of the token and removing access. If that were say, 8,000 API calls needed to remove the token, what happens if a certain percentage of the calls failed? Service Accounts solve this by abstracting these API calls, providing retry logic, and the all important kill switch for a token to instantly remove all of its access.

Better Observability

Pandio took a page out of the Amazon Web Services book and implemented their concept of ARN (Amazon Resource Name) as URN (User Resource Name) to abstract what access policies are associated with users. As you can see from the image below, it becomes much easier to holistically manage who has access to what and what is happening in real-time. Through human readable resource names, one can quickly scan and search to see who can access what. And with these names, an audit trail is easier to keep and investigate. As connections are made and messages are sent, these URN’s are associated with every action to provide an easy to understand picture of Apache Pulsar.

Unique To Pandio

Pandio Service Accounts have powerful UI components as shown above, but this functionality can be accessed programmatically as well. For most enterprises, some form of central authentication and authorization is already setup and configured. And while Pulsar makes these components pluggable so that almost any custom implementation can be made, it still leaves the difficulty of managing these things at scale.

Pandio Service Accounts can be integrated to these custom authentication and authorization components to easily map existing groups and roles to specific users and programmatic users, allowing permissions to be granted automatically without the need to explicitly set each and every permission for each user.

This is an incredible feature that allows enterprises to adopt Apache Pulsar with their own enterprise requirements in record time.