Getting started with the smf rpc framework
25 Mar 2019Cross-posted from https://nwat.xyz/blog/2019/03/22/getting-started-with-the-smf-rpc-framework/.
In this post we are going to take a high-level look at the smf rpc framework, and walk through the process of creating a minimal client/server example. This post will show to setup a build tree, and bootstrap the client and server to exchange messages. We’ll then briefly expand the example to include server-side state management.
The goal of the smf project is provide a high-performance rpc implementation for the Seastar framework. Seastar is a C++ framework for building high-performance server applications. It uses a shared-nothing server design and makes extensive use of modern C++ to provide a futures-based programming model. And while becoming acquainted with Seastar is necessary to build real-world smf-based applications, the content in this post should be understandable by anyone familiar with modern C++.
Here are some helpful links:
Service definition
The smf framework makes extensive use of Google’s flatbuffers serialization
library as a structured container for the network protocol as well as for
defining rpc service interfaces. In this post we are going to be putting
together a basic key/value store service, and we’ll start by examining the
structure of the service in terms of the message types. The following is the
contents of a file called kvstore.fbs that contains the PutRequest message
type holding a single string-based key/value pair. Our service is very simple,
so the response message for a PutRequest is empty—though as we’ll see smf
provides a built-in mechanism for signaling basic return codes.
Finally at the bottom of the file is the rpc_service called MemoryNode
containing a single rpc method Put that takes a PutRequest and responds with
a PutResponse.
namespace kvstore.fbs;
table PutRequest {
key: string;
value: string;
}
table PutResponse {
}
rpc_service MemoryNode {
Put(PutRequest):PutResponse;
}
Once message types and service interfaces are defined a tool in the smf framework is used to perform code generation that produces the C++ implementation of the message types and stub methods for the service interfaces. Next we’ll use the smf-provided cmake tools to invoke code generation within a cmake-based build.
Build scaffolding
Below is the complete CMakeLists.txt that we are going to use to build the
sample project. The bottom three lines import smf as a sub-directory (e.g. via a git
submodule) and configures smf to build and manage all of its dependencies.
cmake_minimum_required(VERSION 3.7.1)
project(smf-kvstore VERSION 0.0.1 LANGUAGES CXX)
set(SMF_MANAGE_DEPS ON CACHE "" INTERNAL FORCE)
set(SMF_BUILD_PROGRAMS OFF CACHE "" INTERNAL FORCE)
add_subdirectory(smf)
Next we use the smfc_gen macro provided by smf to compile the flatbuffers
definition, and store the resulting source files in the kvstore_fbs variable.
smfc_gen(CPP TARGET_NAME kvstore_fbs
OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
SOURCES kvstore.fbs)
And finally we define client and server executables that link against the smf
library and include the source produced by smfc_gen.
link_libraries(smf::smf)
include_directories(PUBLIC ${CMAKE_CURRENT_BINARY_DIR})
add_executable(server server.cc ${kvstore_fbs})
add_executable(client client.cc ${kvstore_fbs})
In the next two sections we’ll work through the creation of the server and the
client. Once the server.cc and client.cc files have been created, the
project can be compiled and tested:
cmake .
make
./server
./client # in another terminal
Note that smf currently builds its dependencies at configure time which is not
a standard technique and may come as a surprise when first starting with the
project. For example, when initially running cmake . to configure the tree,
smf will build Seastar and other dependencies from source. This can take quite a
while to complete, but subsequent builds are fast.
The rpc server
When the flatbuffers file above is compiled it generates a couple C++ header
files. These should be used in the server and client by including
"kvstore.smf.fb.h" (which includes all the other generated files). The
generated code contains the definition of the MemoryNode rpc service that we
saw above.
class MemoryNode: public smf::rpc_service {
private:
std::vector<smf::rpc_service_method_handle> handles_;
public:
virtual ~MemoryNode() = default;
MemoryNode() {
Recall that our service included a single Put method. The stub for this rpc
method is auto-generated. Our implementation will override this default
implementation.
inline virtual
seastar::future<smf::rpc_typed_envelope<kvstore::fbs::PutResponse>>
Put(smf::rpc_recv_typed_context<kvstore::fbs::PutRequest> &&rec) {
using out_type = kvstore::fbs::PutResponse;
using env_t = smf::rpc_typed_envelope<out_type>;
env_t data;
// User should override this method.
//
// Helpful for clients to set the status.
// Typically follows HTTP style. Not imposed by smf whatsoever.
// i.e. 501 == Method not implemented
data.envelope.set_status(501);
return seastar::make_ready_future<env_t>(std::move(data));
}
Now let’s look at our specific server implementation. It starts with a class
that will specialize MemoryNode and implement the rpc methods of our key/value
service.
#include <seastar/core/app-template.hh>
#include <seastar/core/sharded.hh>
#include <smf/rpc_server.h>
#include "kvstore.smf.fb.h"
class kvstore_service final : public kvstore::fbs::MemoryNode {
virtual seastar::future<smf::rpc_typed_envelope<kvstore::fbs::PutResponse>> Put(
smf::rpc_recv_typed_context<kvstore::fbs::PutRequest>&& req) final;
};
Next we provide the implementation of Put. At a high-level this
method does two things. First, it performs any handling for the
request, and second it creates and returns a response. Notice that the
status value is set to 200. Every response in smf contains a helpful
status code that can be used by implementations in any way
they see fit.
Notice also that handling of the request is optional. This is because rpc handlers can be organized into a pipeline within the server, and handlers may decide to not forward the request to downstream handlers.
In any case, this handler doesn’t do much other than print a message with the key and value that are contained in the request. It also prints the identifier of the CPU where this request is being handled. In Seastar-based servers, work is generally sharded across CPUs. For networking applications like smf, Seastar assigns client connections to CPU cores in round-robin assignments to balance load.
seastar::future<smf::rpc_typed_envelope<kvstore::fbs::PutResponse>>
kvstore_service::Put(smf::rpc_recv_typed_context<kvstore::fbs::PutRequest>&& req)
{
if (req) {
LOG_INFO("recv core={} key={} value={}", seastar::engine().cpu_id(),
req->key()->str(), req->value()->str());
}
smf::rpc_typed_envelope<kvstore::fbs::PutResponse> data;
data.envelope.set_status(200);
return seastar::make_ready_future<
smf::rpc_typed_envelope<kvstore::fbs::PutResponse>>(std::move(data));
}
Now to finish up the server. The first important bit is
seastar::sharded<smf::rpc_server> rpc. The sharded object is a special
Seastar object that replicates instances of its template parameter onto each
physical CPU. In this case, we are asking Seastar to replicate the rpc server
across all the cores and to handle connections in parallel.
int main(int args, char **argv, char **env)
{
seastar::app_template app;
app.add_options()
("ip", po::value<std::string>()->default_value("127.0.0.1"), "bind to")
("port", po::value<uint16_t>()->default_value(20776), "listen on")
;
seastar::sharded<smf::rpc_server> rpc;
Next the application itself is started by calling rpc.start(args) which will
setup the service on all cores. Once this bootstrap phase completes the rpc
service established on all cores (via invoke_on_all) will be configured to
serve the kvstore_service that we built above. Finally each instance of the
service is started.
return app.run_deprecated(args, argv, [&app, &rpc] {
seastar::engine().at_exit([&] { return rpc.stop(); });
auto&& cfg = app.configuration();
smf::rpc_server_args args;
args.ip = cfg["ip"].as<std::string>().c_str();
args.rpc_port = cfg["port"].as<uint16_t>();
return rpc.start(args).then([&rpc] {
return rpc.invoke_on_all(
&smf::rpc_server::register_service<kvstore_service>);
}).then([&rpc] {
return rpc.invoke_on_all(&smf::rpc_server::start);
});
});
}
The server can be started using the default arguments by running ./server. You
should see some output similar to the trace below. I’m running this example on a
node with 8 cores and the log reflects this by printing that an rpc server is
starting 8 times. Notice too the log prefix containing the CPU id [shard 0..7].
INFO 2019-03-16 13:26:50,603 [shard 0] smf - rpc_server.cc:73] Starging server:rpc_server{args.ip=127.0.0.1, args.flags=0, args.rpc_port=20776, args.http_port=33140, rpc_routes=rpc_handle_router{rpc_service{name=MemoryNode,handles=rpc_service_method_handle{name=Put}}}, limits=rpc_connection_limits{max_mem:2.000 GB, max_body_parsing_duration: 60000ms, res_avail:2.000 GB (2147483648)}, incoming_filters=0, outgoing_filters=0}
INFO 2019-03-16 13:26:50,603 [shard 0] smf - rpc_server.cc:94] Starting rpc server
INFO 2019-03-16 13:26:50,603 [shard 1] smf - rpc_server.cc:73] Starting server:rpc_server{args.ip=127.0.0.1, args.flags=0, args.rpc_port=20776, args.http_port=33140, rpc_routes=rpc_handle_router{rpc_service{name=MemoryNode,handles=rpc_service_method_handle{name=Put}}}, limits=rpc_connection_limits{max_mem:2.000 GB, max_body_parsing_duration: 60000ms, res_avail:2.000 GB (2147483648)}, incoming_filters=0, outgoing_filters=0}
INFO 2019-03-16 13:26:50,603 [shard 2] smf - rpc_server.cc:73] Starting server:rpc_server{args.ip=127.0.0.1, args.flags=0, args.rpc_port=20776, args.http_port=33140, rpc_routes=rpc_handle_router{rpc_service{name=MemoryNode,handles=rpc_service_method_handle{name=Put}}}, limits=rpc_connection_limits{max_mem:2.000 GB, max_body_parsing_duration: 60000ms, res_avail:2.000 GB (2147483648)}, incoming_filters=0, outgoing_filters=0}
INFO 2019-03-16 13:26:50,603 [shard 3] smf - rpc_server.cc:73] Starting server:rpc_server{args.ip=127.0.0.1, args.flags=0, args.rpc_port=20776, args.http_port=33140, rpc_routes=rpc_handle_router{rpc_service{name=MemoryNode,handles=rpc_service_method_handle{name=Put}}}, limits=rpc_connection_limits{max_mem:2.000 GB, max_body_parsing_duration: 60000ms, res_avail:2.000 GB (2147483648)}, incoming_filters=0, outgoing_filters=0}
INFO 2019-03-16 13:26:50,603 [shard 4] smf - rpc_server.cc:73] Starting server:rpc_server{args.ip=127.0.0.1, args.flags=0, args.rpc_port=20776, args.http_port=33140, rpc_routes=rpc_handle_router{rpc_service{name=MemoryNode,handles=rpc_service_method_handle{name=Put}}}, limits=rpc_connection_limits{max_mem:2.000 GB, max_body_parsing_duration: 60000ms, res_avail:2.000 GB (2147483648)}, incoming_filters=0, outgoing_filters=0}
INFO 2019-03-16 13:26:50,603 [shard 5] smf - rpc_server.cc:73] Starting server:rpc_server{args.ip=127.0.0.1, args.flags=0, args.rpc_port=20776, args.http_port=33140, rpc_routes=rpc_handle_router{rpc_service{name=MemoryNode,handles=rpc_service_method_handle{name=Put}}}, limits=rpc_connection_limits{max_mem:2.000 GB, max_body_parsing_duration: 60000ms, res_avail:2.000 GB (2147483648)}, incoming_filters=0, outgoing_filters=0}
INFO 2019-03-16 13:26:50,603 [shard 7] smf - rpc_server.cc:73] Starting server:rpc_server{args.ip=127.0.0.1, args.flags=0, args.rpc_port=20776, args.http_port=33140, rpc_routes=rpc_handle_router{rpc_service{name=MemoryNode,handles=rpc_service_method_handle{name=Put}}}, limits=rpc_connection_limits{max_mem:2.000 GB, max_body_parsing_duration: 60000ms, res_avail:2.000 GB (2147483648)}, incoming_filters=0, outgoing_filters=0}
INFO 2019-03-16 13:26:50,603 [shard 6] smf - rpc_server.cc:73] Starting server:rpc_server{args.ip=127.0.0.1, args.flags=0, args.rpc_port=20776, args.http_port=33140, rpc_routes=rpc_handle_router{rpc_service{name=MemoryNode,handles=rpc_service_method_handle{name=Put}}}, limits=rpc_connection_limits{max_mem:2.000 GB, max_body_parsing_duration: 60000ms, res_avail:2.000 GB (2147483648)}, incoming_filters=0, outgoing_filters=0}
INFO 2019-03-16 13:26:50,603 [shard 2] smf - rpc_server.cc:94] Starting rpc server
INFO 2019-03-16 13:26:50,603 [shard 3] smf - rpc_server.cc:94] Starting rpc server
INFO 2019-03-16 13:26:50,603 [shard 1] smf - rpc_server.cc:94] Starting rpc server
INFO 2019-03-16 13:26:50,603 [shard 5] smf - rpc_server.cc:94] Starting rpc server
INFO 2019-03-16 13:26:50,603 [shard 4] smf - rpc_server.cc:94] Starting rpc server
INFO 2019-03-16 13:26:50,603 [shard 6] smf - rpc_server.cc:94] Starting rpc server
INFO 2019-03-16 13:26:50,603 [shard 7] smf - rpc_server.cc:94] Starting rpc server
That’s it for the server. Later in this post we’ll expand the implementation so that the server is actually storing the key/value pairs, but right now let’s move on and put together a client that can generate a workload.
Client driver
The client that we’ll walk through below is very basic. It calls the Put rpc
endpoint with a few different key/value pairs. The code resembles the server,
but in this case we do not spin-up a connection per core but rather use a single
connection to keep things simple. Like the server, there a couple arguments that
can change the settings if you happen to run the server on a non-default address
or port.
The most notable difference is that a client object is created. This
is of type MemoryNodeClient and is created when the flatbuffers
definition is compiled.
return app.run(args, argv, [&app] {
auto&& cfg = app.configuration();
auto ip = cfg["ip"].as<std::string>().c_str();
auto port = cfg["port"].as<uint16_t>();
smf::rpc_client_opts opts{};
opts.server_addr = seastar::ipv4_addr{ip, port};
auto client = seastar::make_shared<
kvstore::fbs::MemoryNodeClient>(std::move(opts));
seastar::engine().at_exit([client] { return client->stop(); });
The client’s connect method establishes the connection and returns a future.
When that future is complete we can begin invoking the rpc service methods. Here
we build 10 key/value pairs, and in the next code block we call Put for
each one.
return client->connect().then([client] {
boost::counting_iterator<int> from(0);
boost::counting_iterator<int> to(10);
return seastar::do_for_each(from, to, [client](int i) mutable {
seastar::sstring key = fmt::format("key.{}", i);
seastar::sstring val = fmt::format("val.{}", i);
The PutRequest object is created and populated with the key/value strings,
then the Put method is invoked and the response is logged. After the workload
generator completes, the client and Seastar application is shutdown.
smf::rpc_typed_envelope<kvstore::fbs::PutRequest> req;
req.data->key = std::move(key);
req.data->value = std::move(val);
return client->Put(std::move(req)).then([](auto res) {
if (res) {
LOG_INFO("recv core={} status={}",
seastar::engine().cpu_id(),
res.ctx->status());
}
return seastar::make_ready_future<>();
});
});
}).then([client] {
return client->stop();
}).then([] {
return seastar::make_ready_future<int>(0);
});
});
}
Now we can see the entire thing in action. First start the server, then run the
client a few times. Each time you run the client you should see something like
the following where we receive the response with a code of 200 (see above in
the server with we set the 200 code).
INFO 2019-03-16 08:54:57,229 [shard 0] smf - client.cc:53] recv core=0 status=200
INFO 2019-03-16 08:54:57,229 [shard 0] smf - client.cc:53] recv core=0 status=200
...
Since the client is running as a single thread, you should generally only see it
scheduled on core 0 (i.e. recv core=0 in the log). However, recall that the
server shards its connections across cores. If you examine the log of the
server, you should see that each new connection is handled by a different core.
The first connection is handled on core 0.
INFO 2019-03-16 08:54:42,607 [shard 0] smf - server.cc:20] recv core=0 key=key.0 value=key.0.val
INFO 2019-03-16 08:54:42,607 [shard 0] smf - server.cc:20] recv core=0 key=key.1 value=key.1.val
...
And subsequent connections are assigned to additional cores until all cores are assigned connections and the assignment wraps back around to core 0.
INFO 2019-03-16 08:54:44,816 [shard 1] smf - server.cc:20] recv core=1 key=key.0 value=key.0.val
INFO 2019-03-16 08:54:44,816 [shard 1] smf - server.cc:20] recv core=1 key=key.1 value=key.1.val
INFO 2019-03-16 08:54:44,816 [shard 1] smf - server.cc:20] recv core=1 key=key.2 value=key.2.val
...
INFO 2019-03-16 08:54:56,616 [shard 7] smf - server.cc:20] recv core=7 key=key.0 value=key.0.val
INFO 2019-03-16 08:54:56,616 [shard 7] smf - server.cc:20] recv core=7 key=key.1 value=key.1.val
INFO 2019-03-16 08:54:56,616 [shard 7] smf - server.cc:20] recv core=7 key=key.2 value=key.2.val
...
INFO 2019-03-16 08:54:56,832 [shard 0] smf - server.cc:20] recv core=0 key=key.0 value=key.0.val
INFO 2019-03-16 08:54:56,832 [shard 0] smf - server.cc:20] recv core=0 key=key.1 value=key.1.val
INFO 2019-03-16 08:54:56,832 [shard 0] smf - server.cc:20] recv core=0 key=key.2 value=key.2.val
...
Managing server state
The server we’ve built so far doesn’t do much other than throw the key/value bits from the client on the floor. In this section we are going to modify the server so that the data from the client is stored. We’ll keep it simple and only store the values in memory. This should add just enough detail to our example project to demonstrate how the sharding works when managing server state.
The first thing is to define a new per-core object that will hold
key/value pairs. This class contains a private std::unordered_map to
store client key/value pairs, and a handler for inserting data into
the store. This handler also prints out the core on which it is
running, which will be helpful later when we examine the exact
behavior of the server. Finally, when the server shuts down, we also
print out the contents of the index to examine. We’ll see this in
action after we walk through the rest of the code.
#include <smf/rpc_server.h>
#include "kvstore.smf.fb.h"
+class kvstore_db {
+ public:
+ seastar::future<> put(seastar::sstring key, seastar::sstring value) {
+ LOG_INFO("handling put core={} key={} value={}",
+ seastar::engine().cpu_id(), key, value);
+ store_.insert(std::make_pair(std::move(key), std::move(value)));
+ return seastar::make_ready_future<>();
+ }
+
+ seastar::future<> stop() {
+ for (auto kv : store_) {
+ LOG_INFO("{}/{}: {} {}", &store_,
+ seastar::engine().cpu_id(),
+ kv.first, kv.second);
+ }
+ return seastar::make_ready_future<>();
+ }
+
+ private:
+ std::unordered_map<seastar::sstring, seastar::sstring> store_;
+};
Next we modify the key/value service to track a reference to an instance of the store class we introduced above. Note that both of these will be sharded across the cores in the system. We’ll see shortly how the rpc service is given an instance of the store.
class kvstore_service final : public kvstore::fbs::MemoryNode {
virtual seastar::future<smf::rpc_typed_envelope<kvstore::fbs::PutResponse>> Put(
smf::rpc_recv_typed_context<kvstore::fbs::PutRequest>&& req) final;
+
+ public:
+ kvstore_service(seastar::sharded<kvstore_db>& store) :
+ store_(store)
+ {}
+
+ private:
+ seastar::sharded<kvstore_db>& store_;
};
The other change that is made to the rpc service is to forward the key/value
pair request from the client to the store instance. In order to balance state
across all cores in the system we adopt a simple scheme in which the key is
hashed to a core number. The final piece of this puzzle is the magical
invoke_on method that arranges for a particular method to be executed on a
specific core. In this case, we execute the put operation on the core that is
responsible for the particular key being stored.
@@ -14,6 +44,13 @@ kvstore_service::Put(smf::rpc_recv_typed
if (req) {
LOG_INFO("recv core={} key={} value={}", seastar::engine().cpu_id(),
req->key()->str(), req->value()->str());
+
+ seastar::sstring key = req->key()->str();
+ auto hash = std::hash<seastar::sstring>{}(req->key()->str());
+ auto cpu_id = hash % seastar::smp::count;
+
+ store_.invoke_on(cpu_id, &kvstore_db::put,
+ std::move(key), req->value()->str());
The store is also treated as a sharded object by Seastar, creating an instance per physical core.
@@ -35,10 +72,12 @@ int main(int args, char **argv, char **e
;
}
+ seastar::sharded<kvstore_db> store;
seastar::sharded<smf::rpc_server> rpc;
- return app.run_deprecated(args, argv, [&app, &rpc] {
+ return app.run_deprecated(args, argv, [&app, &store, &rpc] {
seastar::engine().at_exit([&] { return rpc.stop(); });
+ seastar::engine().at_exit([&] { return store.stop(); });
And finally there are some changes to the setup. The piece to pay attention to is that when the rpc service is registered, an instance of the store is passed in which gives it access to the per-core storage service we created at the beginning of this section.
@@ -46,9 +85,12 @@ int main(int args, char **argv, char **e
args.ip = cfg["ip"].as<std::string>().c_str();
args.rpc_port = cfg["port"].as<uint16_t>();
- return rpc.start(args).then([&rpc] {
- return rpc.invoke_on_all(
- &smf::rpc_server::register_service<kvstore_service>);
+ return rpc.start(args).then([&store] {
+ return store.start();
+ }).then([&rpc, &store] {
+ return rpc.invoke_on_all([&store](smf::rpc_server& server) {
+ server.register_service<kvstore_service>(store);
+ });
}).then([&rpc] {
return rpc.invoke_on_all(&smf::rpc_server::start);
});
The client and server are ran in the same way as before, and now we can examine what is happening in the server as the client sends new key/value pairs. As before we log a message when we first receive a request from the client. In this case that connection is handled on core 0. The server then hashes the key and forwards the request to the core that owns that key. In this case the key hashes to core 5 which is where the key/value pair is stored.
INFO 2019-03-17 16:26:02,922 [shard 0] smf - server.cc:51] recv core=0 key=key.0 value=key.0.val
INFO 2019-03-17 16:26:02,922 [shard 5] smf - server.cc:12] handling put core=5 key=key.0 value=key.0.val
The same pattern holds for the rest of the requests which are all handled initially by core 0, and then forwarded to a different core, including core 0 which is also able to store key/value pairs.
INFO 2019-03-17 16:26:02,923 [shard 0] smf - server.cc:51] recv core=0 key=key.1 value=key.1.val
INFO 2019-03-17 16:26:02,923 [shard 6] smf - server.cc:12] handling put core=6 key=key.1 value=key.1.val
INFO 2019-03-17 16:26:02,923 [shard 0] smf - server.cc:51] recv core=0 key=key.2 value=key.2.val
INFO 2019-03-17 16:26:02,923 [shard 6] smf - server.cc:12] handling put core=6 key=key.2 value=key.2.val
INFO 2019-03-17 16:26:02,924 [shard 0] smf - server.cc:51] recv core=0 key=key.3 value=key.3.val
INFO 2019-03-17 16:26:02,924 [shard 1] smf - server.cc:12] handling put core=1 key=key.3 value=key.3.val
INFO 2019-03-17 16:26:02,924 [shard 0] smf - server.cc:51] recv core=0 key=key.4 value=key.4.val
INFO 2019-03-17 16:26:02,924 [shard 5] smf - server.cc:12] handling put core=5 key=key.4 value=key.4.val
INFO 2019-03-17 16:26:02,924 [shard 0] smf - server.cc:51] recv core=0 key=key.5 value=key.5.val
INFO 2019-03-17 16:26:02,924 [shard 3] smf - server.cc:12] handling put core=3 key=key.5 value=key.5.val
INFO 2019-03-17 16:26:02,925 [shard 0] smf - server.cc:51] recv core=0 key=key.6 value=key.6.val
INFO 2019-03-17 16:26:02,925 [shard 5] smf - server.cc:12] handling put core=5 key=key.6 value=key.6.val
INFO 2019-03-17 16:26:02,925 [shard 0] smf - server.cc:51] recv core=0 key=key.7 value=key.7.val
INFO 2019-03-17 16:26:02,925 [shard 0] smf - server.cc:12] handling put core=0 key=key.7 value=key.7.val
INFO 2019-03-17 16:26:02,925 [shard 0] smf - server.cc:51] recv core=0 key=key.8 value=key.8.val
INFO 2019-03-17 16:26:02,925 [shard 6] smf - server.cc:12] handling put core=6 key=key.8 value=key.8.val
INFO 2019-03-17 16:26:02,926 [shard 0] smf - server.cc:51] recv core=0 key=key.9 value=key.9.val
INFO 2019-03-17 16:26:02,926 [shard 7] smf - server.cc:12] handling put core=7 key=key.9 value=key.9.val
We only added 10 key/value pairs in this example, so it’s feasible to dump out the contents of the index on each core. Below is what that dump looks like. We can see that there are 8 different index instances on each of the eight cores. Note that none of the key/value pairs happened to hash to cores 0 or 2.
INFO 2019-03-19 20:44:33,824 [shard 1] smf - server.cc:19] 0x605000011560/1: key.3 val.3
INFO 2019-03-19 20:44:33,824 [shard 3] smf - server.cc:19] 0x606000011790/3: key.5 val.5
INFO 2019-03-19 20:44:33,825 [shard 5] smf - server.cc:19] 0x604000011600/5: key.6 val.6
INFO 2019-03-19 20:44:33,825 [shard 7] smf - server.cc:19] 0x6020000115b0/7: key.9 val.9
INFO 2019-03-19 20:44:33,825 [shard 5] smf - server.cc:19] 0x604000011600/5: key.0 val.0
INFO 2019-03-19 20:44:33,825 [shard 5] smf - server.cc:19] 0x604000011600/5: key.4 val.4
INFO 2019-03-19 20:44:33,825 [shard 6] smf - server.cc:19] 0x6010000115b0/6: key.8 val.8
INFO 2019-03-19 20:44:33,825 [shard 6] smf - server.cc:19] 0x6010000115b0/6: key.1 val.1
INFO 2019-03-19 20:44:33,825 [shard 6] smf - server.cc:19] 0x6010000115b0/6: key.2 val.2
If you found smf or Seastar intriguing I encourage you to check out these projects in more detail. Building high-performance network servers with these two technologies could not be easier or faster. I’m really exciting to keep exploring what they have to offer.
Thanks to Alex for providing feedback on this post!