Thursday, February 20, 2025

Is Vector Search in SQL Server 2025 the Key to Unlocking New Data Insights?

 The advent of AI has ushered in ground breaking changes in most areas of technology. AI is synonymous with a humongous amount of data, data of all types including text, image, audio and video. The databases of the future needs to cater to this changing vista of data and yet compatible with existing technologies.

Microsoft SQL Server has always catered to the technological changes and along its developmental path has introduced many data types to handle diverse data varieties and in recent years has introduced data types such as, "Json', "Stock" and "Geographical" data types. These address  the stock market and the geographical data that accompanies all global operations. 

In addition to data types, it has also changed itself to accommodate not only relational data but also non-relational NoSQL type of data using integration with polyphase technology.

Vector Data Type

SQL Server 2025 has introduced a new **vector data type** designed to store vector data optimized for operations like similarity search and machine learning applications. This is quite a leap for a relational database management system (RDBMS), as it traditionally handles structured data but can handle NoSQL Data.

How is Vector datatype stored in the SQL Server 2025?

A vector in SQL Server is typically stored as a JSON array of floating-point values. Here's an example of how you might define and use a vector in SQL Server:

-- Create a table with a vector column

CREATE TABLE dbo.vectors (

    id INT PRIMARY KEY,

    v VECTOR(3) NOT NULL

);

-- Insert data into the table

INSERT INTO dbo.vectors (id, v) VALUES 

(1, '[0.11, 2, -35]'), 

(2, '[-100.2, 0.0123, 9.876]'

-- Select data from the table

SELECT * FROM dbo.vectors;

This table with two columns has a column of vector data type not null. The vectors are [0.11, 2, -35] and [-100.2, 0.0123, 9.876]

The columns store three dimensional vectors with each dimension stored as a single-precision floating point (float type).

Vector Data Type is not available in 2022

The vector data type is not available in SQL Server 2022. You may have to store vector data type in a JSON column or something customized.

Image Processing and AI

In recent times image processing has been intensely studied. One may ask how to store the dimensional representation of an image. Well, images can have a very high dimensional representation.

Grayscale Image: A 100x100 pixel grayscale image can be represented as a 10,000-dimensional vector (100x100 = 10,000).

Color Image: A 100x100 pixel color image with RGB channels can be represented as a 30,000-dimensional vector (100x100x3 = 30,000).

In SQL Server 2025, the maximum number of dimensions that can be stored using the vector data type is 1998. Each element of the vector is stored as a single-precision (4-byte) floating-point value.

Can image dimensions be stored in SQL Server 2025?

The short answer for storing image dimensions in SQL Server is that it is not possible to store image dimensions without reducing the number of dimensions. There are techniques to reduce the dimensions with various pay offs in quality. We shall discuss them in a later post using a programming language such as python with the use of certain libraries.

Going forward in populating vector data type into the server the SQL Server Integration services can be used to preprocess the image data and then populate the database. This will also be for a future post.


If you are looking for working with databases, stay in and learn from over 15 years of blogging on databases here.  http://hodentekMSSS.blogspot.com

You may also have a look at my database related books:



Saturday, February 1, 2025

Encryption and connecting to SQL Server using SSMS

 This post is describes connecting to SQL Server 2022 using the SQL Server Management Studio and the encryption of connection.

For installing SQL Server 2022 and Microsoft SQL Server Management Studio go here[https://hodentekmsss.blogspot.com/2024/09/installing-sql-server-2022-on-laptop.html],here [https://hodentekmsss.blogspot.com/2024/09/customized-installing-sql-server-2022.html] and here[https://hodentekmsss.blogspot.com/2025/01/install-latest-version-of-sql-server.html].

This post assumes you completed the above two tasks and explores the SQL Server 2022.

Type SQL Server Management Studio App in the search box and click the app in the Search's pop-up.

You should be seeing this image with the Studio interface in the back of the Connect to Server window.

The Server Type:    Database Engine

The Server  Name: Computer name/SQL Server Instance name

Authentication:      Windows Authentication

User name: greyed Computer Name/Computer admin name

These are all the choices you made during installation. Refer to the earlier links in this post.

Connection Security: Early versions did not have this Encryption in this screen and it has three options:

1. Mandatory

2. Optional

3. Strict (SQL Server 2022 and Azure SQL)

This is followed by a checkbox to trust the Server Certificate. Place check mark here and click Connect.

This opens up the Object Explorer, the navigation hub of the SQL Server Management Studio App. 


In earlier versions you would have come across Always Encrypted feature. This feature ensures that sensitive data is always encrypted, even when it is being processed by the SQL Server. It uses client-side encryption to protect data before it is sent to the server, ensuring that the data remains encrypted throughout its lifecycle. 

Mandatory Encryption: This option ensures that the connection to the SQL Server is always encrypted. It uses Transport Layer Security (TLS) to encrypt the data transmitted between the client and the server. This is the default setting in SSMS 20 and later versions.

Optional Encryption: With this option, the connection can be encrypted if the server supports it, but it is not required. If the server does not support encryption, the connection will proceed without encryption. Note: This means that all features of the server can still be accessed, but the data transmitted between the client and the server will not be encrypted, potentially exposing it to security risks.

Strict Encryption: This option is available in SQL Server 2022 and Azure SQL. It enforces the highest level of encryption and does not allow the connection to proceed if the server does not support encryption. This ensures that the connection is always secure.

These encryption options are related to SQL Server encryption in the following ways:

Transport Layer Security (TLS): All three options use TLS to encrypt the data transmitted between the client and the server. TLS is a widely used protocol for securing data in transit and is supported by SQL Server.

Encryption Hierarchy: SQL Server uses a hierarchical encryption and key management infrastructure to protect sensitive data. This includes the use of encryption keys, certificates, and algorithms to secure data at rest and in transit.

The encryption options in SQL Server Management Studio (SSMS) have evolved over time. The "Always Encrypted" feature, which provides client-side encryption to protect sensitive data, was introduced in SQL Server 2016 and became available in SSMS starting with the October 2015 preview version (13.0.700.242) . This feature allows users to encrypt data in their databases without the database system having access to the encryption keys, ensuring that sensitive data remains protected even if the database is compromised.




wyatt from atlas
wyatt from atlas

Mandatory Encryption: This option ensures that the connection to the SQL Server is always encrypted. It uses Transport Layer Security (TLS) to encrypt the data transmitted between the client and the server. This is the default setting in SSMS 20 and later versions.


Optional Encryption: With this option, the connection can be encrypted if the server supports it, but it is not required. If the server does not support encryption, the connection will proceed without encryption. Note: This means that all features of the server can still be accessed, but the data transmitted between the client and the server will not be encrypted, potentially exposing it to security risks.


Strict Encryption: This option is available in SQL Server 2022 and Azure SQL. It enforces the highest level of encryption and does not allow the connection to proceed if the server does not support encryption. This ensures that the connection is always secure.


These encryption options are related to SQL Server encryption in the following ways:


Transport Layer Security (TLS): All three options use TLS to encrypt the data transmitted between the client and the server. TLS is a widely used protocol for securing data in transit and is supported by SQL Server.


Encryption Hierarchy: SQL Server uses a hierarchical encryption and key management infrastructure to protect sensitive data. This includes the use of encryption keys, certificates, and algorithms to secure data at rest and in transit.


The encryption options in SQL Server Management Studio (SSMS) have evolved over time. The "Always Encrypted" feature, which provides client-side encryption to protect sensitive data, was introduced in SQL Server 2016 and became available in SSMS starting with the October 2015 preview version (13.0.700.242) . This feature allows users to encrypt data in their databases without the database system having access to the encryption keys, ensuring that sensitive data remains protected even if the database is compromised.


Tuesday, January 14, 2025

Install the latest version of SQL Server Management Studio 20.2

This is the latest version of SQL Server Management Studio [build no:20.2.30.0, released July 9,2024]. Note: it is not an upgrade to SQL Server 19.x

  • SQL Server Management Studio (SSMS) is an integrated environment for managing any SQL infrastructure, from SQL Server to Azure SQL Database. 
  • SSMS provides tools to configure, monitor, and administer instances of SQL Server and databases. 
  • Use SSMS to deploy, monitor, and upgrade the data-tier components used by your applications and build queries and scripts.

You may download the executable from this link:

https://learn.microsoft.com/en-us/sql/ssms/release-notes-ssms?view=sql-server-ver16

You will find the following (SSMS-ENU) in your download:


You can doble-click the application to start installing. It takes a very short time to install as shown in these screenshots:


Click Install and installation begins.


These are some new features of the SQL Server Management Studio (from the Microsoft documentation).


Connecting to an existing SQL Server 2022:

Kindly follow the links for installing the SQL Server 2022:



If your SSMS 20.2 installation is successful you should be seeing in the Windows Control Panel  several additional components to ensure a smooth and comprehensive experience in using SQL Server Management Studio:

  • Microsoft Visual Studio Tools for Applications 2019: This is necessary for certain features within SSMS that rely on Visual Studio's development environment.
  • SSMS 20.2: The main application for managing SQL Server instances.
  • Microsoft Help Viewer: Provides access to documentation and help files within SSMS.
  • Microsoft OneDrive: This might have been included to facilitate cloud storage and synchronization of your files.
The SSMS20.2 installation also adds the launch program for the SSMS that you can find in the Search as shown here:

C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Microsoft SQL Server Tools.

This folder has the following files: 

Shortcut to the SSMS 20.2 launch
Shortcut to Analysis Services Deployment Wizard 20
Performance tools folder

You can doble click to launch from this location or on the Search results:


The SSMS 20.2 above when clicked launches the application and the SSMS user interface is displayed as shown:


Although the program connects to the installed version of SQL Server it is looking for a trusted connection by way of a certificate. There is a check box below Encryption (that has three options) shown above.

If you do not check this box and try to connect you get the following exception:



If you do place a checm mark saying you trust and then connect you get the following display:


This connection is now OK and the SSMS is OPEN FOR BUSINESS!!!

If you are looking for working with databases, stay in and learn from over 15 years of blogging on databases here.  http://hodentekMSSS.blogspot.com

You may also have a look at my database related books:






 




 








Saturday, January 11, 2025

What is SQLAlchemy?

 SQLAlchemy a toolkit for Python programming language. You can build full blown database applications with it and use the full range of the SQL Language. It is also a Object Relational Mapper(ORM).

SQLAlchemy is what you need (libraries and other tools) for building data-centric apps using Python and using other libraries the possibilities are limitless including AI and Machine Learning.


SQLAlchemy and connecting to data:

The connectors are elements that connect your application to data but in the case of SQLAlchemy they are called 'database dialects'. There are database dialects for each kind of database. These are the most common kinds:


SQLite: A lightweight, disk-based database that doesn’t require a separate server process.

https://hodentekmsss.blogspot.com/2016/09/sqlite-using-visual-studio-community.html


PostgreSQL: A powerful, open-source object-relational database system.

https://hodentekmsss.blogspot.com/search?q=postgresql


MySQL: A widely used open-source relational database management system.

https://hodentekmsss.blogspot.com/2024/11/install-latest-mysql-and-improve.html


Oracle: A proprietary database management system by Oracle Corporation.

https://hodentekmsss.blogspot.com/2013/11/a-collection-of-sql-server-related.html


Microsoft SQL Server: A relational database management system by Microsoft.

https://hodentekmsss.blogspot.com/2024/11/sql-server-2025-ready-to-go.html


MariaDB: A community-developed fork of MySQL.

https://hodentekhelp.blogspot.com/2015/11/how-do-you-access-mariadb-in-xampp.html

These connectors make it easy to connect SQLAlchemy to various database systems, ensuring compatibility and flexibility in developing data-centric applications.

If you are looking for working with databases, stay in and learn from over 15 years of blogging on databases here.

You may also have a look at my database related books:





You may


Is Vector Search in SQL Server 2025 the Key to Unlocking New Data Insights?

 The advent of AI has ushered in ground breaking changes in most areas of technology. AI is synonymous with a humongous amount of data, data...