Wednesday, August 3, 2022

Storing and Processing Big Data with SQL Server

  Here is a reasonable article comparing SQL vs. NoSQL. Here you can also look up the differences between RDBMS and Document Databases. 

https://phoenixnap.com/kb/sql-vs-nosql

While Amazon has its own DocumentDB, MongoDB is used in a lot of places (Forbes, Toyota, etc) and Amazon's DocumentDB is compatible with MongoDB. 

Of course, Microsoft's SQL Server is a mature product and it can even handle BigData using Polybase virtualization. You can query data from any SQL Server, Oracle, Teradata, MongoDB, and other data sources using external tables. 

https://docs.microsoft.com/en-us/sql/big-data-cluster/big-data-options?view=sql-server-ver16

Connectivity to HDFS now uses published REST APIs instead of the Java Hadoop client. all you need to do is to configure connectors while configuring the AZURE Storage.

Here is a schematic from Microsoft's documentation of BigData storage and processing in the Microsoft platform.


Also, storing data in itself is not sufficient and Microsoft has POWER BI which also visualization of data from a huge number of database products. It is hard to beat Microsoft at this game.

I am somewhat slanted towards Microsoft due to my association with Microsoft database products for a long time. I have not received any remuneration from Microsoft for this post.

SQL Server 2025 ready to go

 I have not yet done looking at SQL Server 2022, SQL Server 2025 is ready to go. Microsoft is indeed relentless!  Microsoft announced SQL Se...