Changing Database Markets

In this article, I am going to look at the changing face of database technology and how we, as service providers, need to adapt to these new technologies, if we are to continue to provide the best service to our clients.

On the internet and through social media and entertainment apps, the use of data has changed over the past few years. We expect, as end users, that our experience of a company is tailored to us. Take for example Amazon, Google and even Wimbledon. This has led to a change in what data is stored and how it is used.

This change is now filtering into other aspects of our working lives.

Businesses want to get more value from their systems allowing them to market more effectively and provide a better service to their clients. Traditional database systems could not easily provide this detail, so new ways were needed to store and manage data. But this has a down side for the client – how do they choose which type of database to use?

Wikipedia has the following to say regarding databases;

What is the main purpose of a database?

“A database is an organized collection of data. It is the collection of schema’s, tables, queries, reports, views and other objects. The data are typically organized to model aspects of reality in a way that supports processes requiring information, such as modelling the availability of rooms in hotels in a way that supports finding a hotel with vacancies.” Wikipedia, 2016

The ability to store any data that is pertinent to my business in a way that makes sense to me.

HOW DO WE, AS USERS, INTERACT WITH A DATABASE?

“A database management system (DBMS) is a computer software application that interacts with the user, other applications, and the database itself to capture and analyze data. A general-purpose DBMS is designed to allow the definition, creation, querying, update, and administration of databases.” – Wikipedia, 2016

The ability to get to query, modify and report on my data whenever I want.

HOW MANY DIFFERENT DBMS’S ARE THERE OUT THERE?

“Well-known DBMSs include MySQL, PostgreSQL, Microsoft SQL Server, Oracle, Sybase, SAP HANA, and IBM DB2.” – Wikipedia, 2016

There are now hundreds of databases entering the market place all supporting different databases models.

WHAT ARE THE DIFFERENT MODELS?

“A database is not generally portable across different DBMSs, but different DBMSs can interoperate by using standards such as SQL and ODBC or JDBC to allow a single application to work with more than one DBMS. Database management systems are often classified according to the database model that they support; the most popular database systems since the 1980s have all supported the relational model as represented by the SQL language.” – Wikipedia, 2016

Common logical data models include:

  • Hierarchical database
  • Network
  • Relational
  • Object
  • Document
  • Flat file
  • Multidimensional
  • XML
  • Graph
  • Spatial

image1

Diagram adapted from a 2012 diagram conceived by 451 Group

The above diagram is by no means a complete list but serves to illustrate just how many different database engines exist in the market and there are many more in development and in beta. Choosing the right database for your workload may not be as easy as you first thought.

WHICH DATABASE MODEL IS CORRECT FOR MY BUSINESS?

image2

This question is being asked more and more. Which database system should I choose is now secondary to which database model best suits my purpose. Only when you understand the workloads that you intend to throw at the database engine can you make any decision as to which database model is suitable.

EXISTING COMPANIES ARE ASKING – “HAVE WE CHOSEN THE CORRECT DATABASE MODEL FOR OUR GROWING WORKLOADS?”

NEW COMPANIES ARE ASKING – “WHICH DATABASE MODEL IS BEST FOR ME WHEN I PRESENTLY DO NOT KNOW WHAT MY DATABASE WORKLOAD WILL LOOK LIKE?”

Successfully answering the above question will determine the level of success achieved from your chosen database strategy.

Conceivably, any requirement can be achieved with any database platform, given enough time, and the correct resources. We can manipulate any database engine to achieve any workload and can list several clients that have, historically, been doing just this.

The result of this approach simply leads us back to hammering the square block into the circular hole.

My argument is that no single database model can fully represent all of the requirements of a growing company that relies on their database for:

  • OLTP
  • Analytics
  • Batch
  • Fraud detection
  • Customer retention
  • Sales
  • Inventory management
  • …

Taking into account the uses listed above we could easily employ relational, document and graph models to provide the best user experience. The choice now becomes whether to settle for the best, or closest, fit or to accept that a single DBMS may no longer suffice. Now we know what workloads we have in our business it will be easy to select the correct database engine – OR WILL IT?

HOW DO I KNOW WHICH MODEL IS BEST FOR ME?

Here are the definitions for some of the most common database models being employed today:

HIERARCHICAL MODEL

image4

This model organizes data into a tree structure using both parent and child data segments. This structure implies that a parent can have many children. Data is stored in a series of records, which have a set of field values. All instances of a specific record are classed as a record type, equivalent to a table in a relational model. To create links between these record types, the hierarchical model uses Parent Child Relationships.

Developed by IBM and utilized in a number of mainframe based applications, this model is still widely used in applications requiring high performance and availability. Banking and Telecoms still rely on this database model today. One of the most popular uses of the hierarchical databases is the applications IBM Information Management System (IMS), Windows Registry and the Windows OS.

The main limitation of this model is that it is confined to a “one to many” relationship.

RELATIONAL MODEL

image5

Most people are familiar with this model and this familiarity has led to the current RDBMS dominance in the market.

This model allows:

  • Definition of data structures
  • Storage and retrieval operations
  • Integrity constraints.

The data and relations between them are organized in tables. A table is a collection of records and each record in a table contains the same fields.

Fields within a table can be designated as primary keys and indexes may be allocated to keys and other fields within the table to speed up access. Where a relationship exists between 2 tables a join operation is used to select the related records by matching the field values.

Most applications today have been designed utilizing this kind of model. This is mainly due to the fact that data is only stored once which reduces the amount of storage required. For instance, 3 people residing at the same house would be represented by 3 separate records identifying each person but only 1 record identifying the address.

The flexibility of the model also makes it very easy to develop new applications and to create schema’s that accommodate the requirements. Most OLTP systems today utilize this database model.

OBJECT MODEL

This model adds new object storage capabilities to the relational systems including complex objects such as time-series and geospatial data and binary media such as audio, video and images.

The main benefits of this is that an ORDBMS server can execute complex analytical and data manipulation operations to search and transform multimedia and other complex objects.

GRAPH MODEL

image6

A graph database is a database that uses graph structures for queries with nodes. This contrasts with conventional relational databases, where links between data are based on the data itself, and related items are gathered by searching for this data within the store. Graph databases are designed to allow simple and rapid retrieval of complex hierarchical structures.

Graph databases provide firms with new insights and intelligence that were extremely challenging to produce with traditional technologies such as relational or XML databases. Graph databases can store and process large volumes of data and enable the search, discovery, and exploration of large networks. Graph databases are extremely useful in the following situations:

  • Dynamic systems where the data topology is difficult to predict
  • Dynamic requirements that evolve with the business
  • Problems where the relationships in data contribute meaning & value

Today this model is seeing great success in the following use cases:

  • Real time routing and tracking
  • Social Networking
  • Fraud Analytics
  • Identity and Access
  • Recommendations
  • Network and IT operations

DOCUMENT MODEL

image7

In this model each record and its associated data is thought of as a document.

All related data is stored together in a single document. Each document is seen as an independent unit. The data is unstructured which means that it can be stored very easily.

Storing data in this way means that it can be read contiguously from disk which improves performance. Unstructured data is perfect for this model.

This model is being used in Retail, Government and Finance sectors for:

  • Large data management and analytics
  • Personalization
  • Data aggregation
  • Social networking

MAKING THE DECISION

It can be very difficult to decide which database model best suits your requirements and then to select a specific DBMS especially as every database vendor you speak to will tell you that their solution is “the only solution” for your requirements.

Remember that database vendors want to promote their products above and beyond all others. In some instances, a proof of concept project might be required to determine which database engine performs best against your workload. Most database vendors are open to this approach and will grant access to their software for testing purposes.

Some customers are led to an initial decision based on an application they wish to roll out. Historically once this decision was made the chosen database platform was then deployed across the business and all workloads were massaged into that database engine. This played very much into the “Relational” database model’s hands due to its flexibility – it became the default choice.

This is where the database market is seeing the most change recently. Just because a point of sale application requires SQL Server is no longer driving companies to role SQL Server out across their company as the standard database choice. More and more we are meeting customers with multiple database engines performing different tasks across their business. This is where the database industry is heading with key vendors producing outstanding platforms adhering to a single database model that is more suited to specific workloads.

IS THIS THE BEST APPROACH AND THE BEST WAY TO REPRESENT OUR CLIENTS?

Businesses choose service providers that can solve their immediate problems. We must shift our focus to delivering solutions because the market place has changed. Clients no longer wish to purchase that square block to fill their circular hole. Service providers must adapt to meet the changing demands of their clients.

More and more we are being approached to examine particular workloads within our client’s environments; Areas where performance has been a significant hurdle, or capacity is becoming a concern. Historically a database administrator would immediately set about tuning the existing database to the Nth degree to improve performance and ensure that more capacity would not degrade the end user experience.

We are presently running 3 proof of concept projects for customers with very different requirements. This is by far the best way for a customer to ensure they are getting the database product to manage their workloads.

IN SUMMARY

With all of this in mind surely we must conclude that every single database related workload must be assessed to determine which database model is best suited to the task. Once the database model is agreed on then we must examine the different database systems that support that database model, and select the most appropriate engine for the individual customer.

A client might end up with several database systems working together to deliver their requirements. Businesses need to be reminded that they have a lot of choice and there is absolutely no need to settle for the closest fit solution. With a little research the perfect “blend” of databases can be established. Still confused by all the options available or unsure how to proceed?

Contact RDB Concepts and our team of friendly, highly experienced certified professionals will be able guide you through your decision making steps, provide the right level of assistance to ensure the service is delivered in a low risk \ stress free manner via our technical delivery, proof of concept services.

Contact Us