The Secret to Horizontal Scalability of Any System
If asked about the secret to scalability of any system, the simplest answer is “a straight vertical line”.
This answer assumes a tiered system such as shown in illustration 1, in which case it describes the
shortest access path from the user to the data or more precisely from the place where the data is stored
to the place where the data is actually processed. The latter is the basis for further simplifications:
1) The delivery of the result set to the user connected to the application layer (the application user) is
typically not a concern. 2) Whether a user connects directly to the database management system
(DBMS) or via a connection pool is merely a consideration to prevent contention rather than an access
path concern; the per-connection access path to the data is typically the same from the perspective of
the DBMS. In case of the Oracle Relational DBMS (from here on referred to as the “Oracle
Database”), data is requested by a session connected to an instance regardless of whether the user
connects directly to the database or uses a connection pool.
With these simplifications in mind, the actual access path needs to be considered. Most DBMSs use an
indirect access path to data these days. “Indirect” means that the client (user) does not access the data
files directly, but connects to a management layer, which is used to retrieve the data on behalf of the
user, often providing additional capabilities such as caching functionality. In the concrete case of the
Oracle Database, the client is connected to the Oracle Database Instance (the set of processes and
memory allocated on a server), which will access the data files stored on disk (the database). Using
these definitions of “instance” and “database” for this paper going forward, a multi-instance database
represents a database, in which more than one instance (on more than one server) connects to the
database stored on concurrently accessed shared storage (shared disk approach). This is also the basic
definition of an Oracle RAC database, which will be used later on.
In order to horizontally scale such systems, all that is required is to add instances (on additional
servers) to the system. Whether the system uses a shared disk approach as described or uses a shared
nothing approach, in which case each instance may actually only be responsible for a subset of data,
would not matter in this case, as long as the DBMS as a whole guarantees (by whatever means) that
the data that is requested by a user can transparently be delivered to the instance, to which the user is
connected. The art, and hence the secret to scalability of such a system, would then be the way the
DBMS routes the user request to the best possible instance to operate on the data.
Illustration 2: Three Alternative Architectures - The Same Idea Applies