Communicating with MASQNode

Using Command-Line is acting as a UI to configuring, communicate and start/stop the MASQ Node software. These communications are directed to the MASQNode daemon service.

Communication Between MASQNode and User Interfaces

Background

Project Architecture

The MASQNode (or MASQNode.exe for Windows) binary is used for two different purposes. One is called the Daemon; the other is called the Node.

The Node contains all the communications capabilities MASQ is known for. Its job is to start with root privilege, open low ports, drop privilege to user level, and settle into sending and receiving CORES packages.

The Daemon is different. Its job is to start when the machine boots, with root privilege, and keep running with root privilege until the machine shuts down. It is not allowed to communicate over the Internet, or with the Node. This reduces the chance that an attacker's hack of the Node could gain root privilege on a user's machine.

Since the Daemon is always running, it listens on a localhost-only port (5333 by default) for connections from user interfaces. UIs connect first to the Daemon on its well-known port. There are certain conversations that the Daemon can carry on with the UI (one of which tells the Daemon to start up the Node), but when it's time, the Daemon will tell the UI where the Node is so that the UI can connect directly to the Node.

If the Node crashes, the UI should reconnect to the Daemon. From there, if desired, it can direct the Daemon to restart the Node.

Any number of UIs can connect to the Daemon and the Node. Information that is relevant only to one UI is sent only to that UI; information that is relevant to all is broadcast. Currently there is no way for a UI to subscribe only to those broadcasts in which it is interested; it will receive all broadcasts and has the responsibility to ignore those it doesn't care about. If necessary, the subscription functionality can be added to the Node in the future.

Communications Architecture

Level 1

If the Daemon is started without specific settings, like this

$ ./MASQNode --initialization

it will try to come up listening for UI connections on port 5333. But if it's started like this

$ ./MASQNode --initialization --ui-port 12345

it will try to come up listening for UI connections on port 12345. If it finds the target port already occupied, it will fail to start.

The Node is started by the Daemon. When the Daemon starts the Node, it will choose an unused port and direct the Node to listen for UIs on that port. When the Daemon redirects a UI to the Node, it will supply in the redirect message the port on which the Node is running.

The Daemon and the Node listen for UIs only on the localhost pseudo-NIC. This means that all the UIs for a particular Daemon or Node must run on the same computer as the Daemon or Node: they cannot call in over the network from another machine. This restriction is in place for security reasons.

Level 2

The link between the UIs and the Daemon or Node is insecure WebSockets, using the protocol name of MASQNode-UIv2. Any other protocol name will be rejected, and no connection will be made.

Level 3

Once the WebSockets connection is established, all the messages passed back and forth between the UIs and the Daemon or Node are formatted in JSON. A message packet is always a JSON object, never a scalar or an array.

Level 4

The low-level JSON format of MASQNode-UIv2 messages is reasonably simple. It looks like this:

{
    "opcode": <string>,
    "contextId": <positive integer>,
    "payload": <optional object>,
    "error": <optional object>
}

The opcode is a short string that identifies the message type. If a message is a request (UI to Node) and the protocol dictates that a response (Node to UI) should result from it, both the request and the response will have the same opcode.

The contextId is a positive integer best thought of as a conversation number. Just as there can be many UIs connected to the same Node, each UI can be carrying on many simultaneous conversations with the Node. When a request is sent as part of a unique conversation, the Daemon and the Node guarantee that the next message received in that conversation will be the response to that request. It is the responsibility of each UI to manage contextIds. When the UI wants to start a new conversation, it merely mentions a new contextId in the first message of that conversation; when it's done with a conversation, it just stops mentioning that conversation's contextId.

It may be tempting to use a single contextId for all the messages a UI sends in its lifetime, and this is perfectly legal as far as the Node and Daemon are concerned; but if the UI does this, it will have to determine for itself which conversation each incoming message is part of. For example, if there are three conversations going on at once, this might happen:

1. → Request for conversation 1

2. → Request for conversation 2

3. ← Response for conversation 1

4. → Request for conversation 3

5. ← Broadcast from Node

6. ← Response for conversation 3

7. ← Response for conversation 2

If each conversation has its own ID, it'll be a lot easier to tell what's going on when a message arrives than it will be if every message is part of conversation 555.

At the other extreme, a UI may choose to start a new conversation for every request/response pair. This is fine.

Some messages are always isolated, and never part of any conversation, like the Broadcast in step 5 above. These messages will be identifiable by their opcode, and their contextId should be ignored. (In the real world, it's always zero, but depending on that might be dangerous.)

Neither the Daemon nor the Node will ever start a conversation, although they will send isolated, non-conversational messages.

The payload is the body of the message, with its structure being signaled by the contents of the opcode field. See the Message Reference section below for specifics about the payload field for each type of message. It will be present if and only if the error field is not present.

The object in the error field, if present, tells about the error that was encountered in the process of trying to satisfy a request. It will be present if and only if the payload field is not present. It will have this structure:

{
    code: <nonnegative integer>,
    message: <string>
}

The code field is a 64-bit integer. Its numeric value is not particularly important, but it denotes a kind of error. The UI can tell whether a particular operation is producing the same kind of error repeatedly, or different kinds of errors, by comparing one code to the next.

The message field is a string with a hopefully-friendly description of the error.

There is no provision in the MASQNode-UIv2 protocol for UIs to communicate with one another. A UI may be able to deduce, from broadcasts, the existence of other UIs, but it can never be assured that there aren't any other UIs connected to the Node or Daemon.

Level 5

The structure of the payload of a MASQNode-UIv2 message depends on the opcode of that message. See the Message Reference section below.

General Operational Concepts

Daemon

Setup

The Node requires quite a bit of configuration information before it can start up properly. There are several possible sources of this configuration information. The primary source, though, is the command line that's used to start the Node. There are many parameters that can be specified on that command line, and the Daemon needs to know them all in order to start the Node.

Accumulating this information is the purpose of the Daemon's Setup functionality, which is a large proportion of what it does.

The Daemon has a space inside it to hold Setup information for the Node. A UI can query the Daemon to get a dump of the information in the Setup space. When the Node is not running, the information in the Setup space can be changed by the UI. When the Node is running, the information in the Setup space is frozen and immutable. This is so that when the Node is running, you can use the UI to query the Daemon to discover the configuration with which the Node was started.

If a Node is shut down, a new Node can easily be started with exactly the same configuration as its predecessor as long as the information in the Setup space is not disturbed.

Start

When the Start operation is triggered, the Daemon will try to start the Node with the information in the Setup space. The response message will tell whether the attempt succeeded or failed.

Redirect

As long as the UI sends the Daemon messages that the Daemon understands, the Daemon will respond appropriately to them. But if the UI sends the Daemon a message the Daemon doesn't understand, the Redirect operation may come into play.

If the Node is not running, there's nowhere to Redirect, so the Daemon will just send back an error response.

However, if the Node is running, the Daemon will send back a Redirect response, which will contain both information about where the Node is running and also the unexpected message sent to the Daemon. When the UI gets a Redirect, it should drop the WebSockets connection to the Daemon, make a WebSockets connection to the Node on the port supplied in the Redirect message (on localhost, using the MASQNode-UIv2 protocol), and resend the original message--which, in case the UI doesn't remember it anymore, is helpfully included in the Redirect payload. If it's a valid Node message, the Node should respond appropriately to it.

Node

Database password

The Node stores its configuration information in a database. A UI should certainly never attempt to write to this database, but it also shouldn't attempt to read from it, for two reasons: first, some of the information in the database is encrypted because it's sensitive; and second, the Node does some caching work for performance reasons, so what a UI finds in the database might be several minutes or more old. The UI should ask the Node directly for the information it needs.

The information in the database that's encrypted needs a password to decrypt it. When the Node is first installed, there is no secret information in the database; therefore, the database has no password. A password can be set on the database without storing any secrets in it, if desired, but in order to store secrets, a password must be set on the database.

The password is never stored anywhere but in memory by the Node; it should not be persisted anywhere by a UI either. In order to carry out certain instructions, the Node will need the password from the UI, which means the UI will need to get it from the user.

Using MASQNode-UIv2 messages, the UI can check to see if a password is correct; it can change the database password (if it knows the old one); and it can be notified when some other UI changes the password (so that it knows the one it's aware of is no longer valid).

Configuration

The configuration information with which the Node runs (which is different from the setup information with which the Daemon starts a Node) is available via MASQNode-UIv2 as well. A UI can request the configuration information, and if the information changes for some reason, all UIs will be notified so that--if desired--they can request the latest version.

Shutdown

The Shutdown operation causes the Node to cease operations and terminate. The UI will receive a response, and then the WebSockets connection will be dropped by the Node.

Whenever the WebSockets connection is dropped, whether the Shutdown operation is in progress or not, the UI should reconnect to the Daemon.

If for some reason the WebSockets connection is not dropped by the Node within a few milliseconds of the response to the Shutdown message, that indicates that the Node has somehow become hung on the way down. In this case, the WebSockets connection to the Node will probably be of no further use. The UI may choose to inform the user that bad things are happening which will probably require user intervention.