Openstack4j Multi-thread Token Sharing

Author: slankka

Country/Region: China Mainland

Languages skills: English but not well

Keywords

• Cloud Virtual Machine
• Paas
• Openstack
• Multi-threading

Background

We are building a Virtual Machine Platform as Paas, these days I'm facing a challenge to access multiple openstack clusters.

The disadvantage of Openstack4j:

• lacking of automatic token renewer
• binding token in ThreadLocal

which will cause token sharing problem in multi-thread senario.

Solution found

Here is Openstack official solution:
Using the Same Client Between Threads

the key feature is:

OSFactory.clientFromToken(token);

I got inspired by this example: we can obtain an exists token, and create a new client in my own thread.

Design and implementation

This is a design of mine. It finally solved problems above.

I'm using functional programming and simple threadpool of java core technique.

The picture demostrates these parts from left to right:
Thread sources: UI actions are made by users, or some automatic framework such as cron scheduler.
Service Layer:Actions are converted to Functional objects such as Runnable instances.
Client Proxy: Execute those Runnable objects asynchronously.

• Create a new client based on that token.
• Execute and wait for Future(result).

Openstack Contract: Consists of TokenManager and Scheduled Renewer.
TokenManager is responsible for managing multiple clusters token. The key concept is using ThreadPool's core thread to store and retrieve Token. It exposes three public methods and a cluster specific token sharing storage.

• ensureOpen() authenticate and re-authenticate when expired
• getClient() create client object
• checkToken() check expiration

Make sure all method calling of above are invoked under different thread.

Scheduled Renewer is responsible for renew token when expired. It manages an SingleThreadScheduledExecutor for keeping token available, and same amount of clusters of authenticating threads, which are implemented by fixedThreadPool.

• initialize first connection, scheduleWithFixedDelay.
• Call ensureOpen periodically then update token sharing storage.

It's fine that a authenticating thread execute and finished, because it is also a core thread and alive, the token are always stored in ThreadLocal of its.

Lifecycle of thread management are omitted

And example:
Example:

Function<OSClient.OSClientV3, ActionResponse> action = (x) -> x.blockStorage().volumes().get(uuid);

//Isoliated from UI thread
pool.submit( ()->{
    OSClient.OSClientV3 client = getClient(); //Provided by Openstack Contract
    return client.apply(action);
    });