When accessing a remote memory, the processor must communicate with the
memory through the network. Figure is a simplified diagram
of the data path on a single node.
Figure: Single-node architecture
When the processor performs a read and the requested data is not
in the cache or
the local memory, the
node interface constructs a request packet and
stores it in the queue until an empty slot is detected on the ring.
The first empty slot is filled with the request packet.
When the remote node receives the request, it either
accepts or rejects it. The request is rejected if the node
interface is already busy satisfying another request.
If this occurs,
the request slot is marked with a negative acknowledgement, and
the packet is returned to the sender.
Upon receiving the negative acknowledgement, the requesting node
will retry.
Once the request is accepted, it will pass through the node
interface-memory buffer, and be sent to the memory controller. The memory
controller then reads the data from its memory, and sends it to the node
interface. The node interface packs the data into either one
or two packets, and stores them
in queue .
When the requesting node interface receives the data,
the data is sent to the cache.
To understand the purpose of the cache-memory buffer and the node interface-memory buffer, consider two processors A and B. If processor A performs a local memory access, while processor B requests data from node A, then the memory unit on node A might receive a request from the node interface (to satisfy B's request) at the same time that it receives a request from the cache (A's request). The buffers hold the two requests, and the memory control unit satisfies each in a round-robin fashion.