Thursday, February 20, 2025

Is Vector Search in SQL Server 2025 the Key to Unlocking New Data Insights?

 The advent of AI has ushered in ground breaking changes in most areas of technology. AI is synonymous with a humongous amount of data, data of all types including text, image, audio and video. The databases of the future needs to cater to this changing vista of data and yet compatible with existing technologies.

Microsoft SQL Server has always catered to the technological changes and along its developmental path has introduced many data types to handle diverse data varieties and in recent years has introduced data types such as, "Json', "Stock" and "Geographical" data types. These address  the stock market and the geographical data that accompanies all global operations. 

In addition to data types, it has also changed itself to accommodate not only relational data but also non-relational NoSQL type of data using integration with polyphase technology.

Vector Data Type

SQL Server 2025 has introduced a new **vector data type** designed to store vector data optimized for operations like similarity search and machine learning applications. This is quite a leap for a relational database management system (RDBMS), as it traditionally handles structured data but can handle NoSQL Data.

How is Vector datatype stored in the SQL Server 2025?

A vector in SQL Server is typically stored as a JSON array of floating-point values. Here's an example of how you might define and use a vector in SQL Server:

-- Create a table with a vector column

CREATE TABLE dbo.vectors (

    id INT PRIMARY KEY,

    v VECTOR(3) NOT NULL

);

-- Insert data into the table

INSERT INTO dbo.vectors (id, v) VALUES 

(1, '[0.11, 2, -35]'), 

(2, '[-100.2, 0.0123, 9.876]'

-- Select data from the table

SELECT * FROM dbo.vectors;

This table with two columns has a column of vector data type not null. The vectors are [0.11, 2, -35] and [-100.2, 0.0123, 9.876]

The columns store three dimensional vectors with each dimension stored as a single-precision floating point (float type).

Vector Data Type is not available in 2022

The vector data type is not available in SQL Server 2022. You may have to store vector data type in a JSON column or something customized.

Image Processing and AI

In recent times image processing has been intensely studied. One may ask how to store the dimensional representation of an image. Well, images can have a very high dimensional representation.

Grayscale Image: A 100x100 pixel grayscale image can be represented as a 10,000-dimensional vector (100x100 = 10,000).

Color Image: A 100x100 pixel color image with RGB channels can be represented as a 30,000-dimensional vector (100x100x3 = 30,000).

In SQL Server 2025, the maximum number of dimensions that can be stored using the vector data type is 1998. Each element of the vector is stored as a single-precision (4-byte) floating-point value.

Can image dimensions be stored in SQL Server 2025?

The short answer for storing image dimensions in SQL Server is that it is not possible to store image dimensions without reducing the number of dimensions. There are techniques to reduce the dimensions with various pay offs in quality. We shall discuss them in a later post using a programming language such as python with the use of certain libraries.

Going forward in populating vector data type into the server the SQL Server Integration services can be used to preprocess the image data and then populate the database. This will also be for a future post.


If you are looking for working with databases, stay in and learn from over 15 years of blogging on databases here.  http://hodentekMSSS.blogspot.com

You may also have a look at my database related books:



Saturday, February 1, 2025

Encryption and connecting to SQL Server using SSMS

 This post is describes connecting to SQL Server 2022 using the SQL Server Management Studio and the encryption of connection.

For installing SQL Server 2022 and Microsoft SQL Server Management Studio go here[https://hodentekmsss.blogspot.com/2024/09/installing-sql-server-2022-on-laptop.html],here [https://hodentekmsss.blogspot.com/2024/09/customized-installing-sql-server-2022.html] and here[https://hodentekmsss.blogspot.com/2025/01/install-latest-version-of-sql-server.html].

This post assumes you completed the above two tasks and explores the SQL Server 2022.

Type SQL Server Management Studio App in the search box and click the app in the Search's pop-up.

You should be seeing this image with the Studio interface in the back of the Connect to Server window.

The Server Type:    Database Engine

The Server  Name: Computer name/SQL Server Instance name

Authentication:      Windows Authentication

User name: greyed Computer Name/Computer admin name

These are all the choices you made during installation. Refer to the earlier links in this post.

Connection Security: Early versions did not have this Encryption in this screen and it has three options:

1. Mandatory

2. Optional

3. Strict (SQL Server 2022 and Azure SQL)

This is followed by a checkbox to trust the Server Certificate. Place check mark here and click Connect.

This opens up the Object Explorer, the navigation hub of the SQL Server Management Studio App. 


In earlier versions you would have come across Always Encrypted feature. This feature ensures that sensitive data is always encrypted, even when it is being processed by the SQL Server. It uses client-side encryption to protect data before it is sent to the server, ensuring that the data remains encrypted throughout its lifecycle. 

Mandatory Encryption: This option ensures that the connection to the SQL Server is always encrypted. It uses Transport Layer Security (TLS) to encrypt the data transmitted between the client and the server. This is the default setting in SSMS 20 and later versions.

Optional Encryption: With this option, the connection can be encrypted if the server supports it, but it is not required. If the server does not support encryption, the connection will proceed without encryption. Note: This means that all features of the server can still be accessed, but the data transmitted between the client and the server will not be encrypted, potentially exposing it to security risks.

Strict Encryption: This option is available in SQL Server 2022 and Azure SQL. It enforces the highest level of encryption and does not allow the connection to proceed if the server does not support encryption. This ensures that the connection is always secure.

These encryption options are related to SQL Server encryption in the following ways:

Transport Layer Security (TLS): All three options use TLS to encrypt the data transmitted between the client and the server. TLS is a widely used protocol for securing data in transit and is supported by SQL Server.

Encryption Hierarchy: SQL Server uses a hierarchical encryption and key management infrastructure to protect sensitive data. This includes the use of encryption keys, certificates, and algorithms to secure data at rest and in transit.

The encryption options in SQL Server Management Studio (SSMS) have evolved over time. The "Always Encrypted" feature, which provides client-side encryption to protect sensitive data, was introduced in SQL Server 2016 and became available in SSMS starting with the October 2015 preview version (13.0.700.242) . This feature allows users to encrypt data in their databases without the database system having access to the encryption keys, ensuring that sensitive data remains protected even if the database is compromised.




wyatt from atlas
wyatt from atlas

Mandatory Encryption: This option ensures that the connection to the SQL Server is always encrypted. It uses Transport Layer Security (TLS) to encrypt the data transmitted between the client and the server. This is the default setting in SSMS 20 and later versions.


Optional Encryption: With this option, the connection can be encrypted if the server supports it, but it is not required. If the server does not support encryption, the connection will proceed without encryption. Note: This means that all features of the server can still be accessed, but the data transmitted between the client and the server will not be encrypted, potentially exposing it to security risks.


Strict Encryption: This option is available in SQL Server 2022 and Azure SQL. It enforces the highest level of encryption and does not allow the connection to proceed if the server does not support encryption. This ensures that the connection is always secure.


These encryption options are related to SQL Server encryption in the following ways:


Transport Layer Security (TLS): All three options use TLS to encrypt the data transmitted between the client and the server. TLS is a widely used protocol for securing data in transit and is supported by SQL Server.


Encryption Hierarchy: SQL Server uses a hierarchical encryption and key management infrastructure to protect sensitive data. This includes the use of encryption keys, certificates, and algorithms to secure data at rest and in transit.


The encryption options in SQL Server Management Studio (SSMS) have evolved over time. The "Always Encrypted" feature, which provides client-side encryption to protect sensitive data, was introduced in SQL Server 2016 and became available in SSMS starting with the October 2015 preview version (13.0.700.242) . This feature allows users to encrypt data in their databases without the database system having access to the encryption keys, ensuring that sensitive data remains protected even if the database is compromised.


Is Vector Search in SQL Server 2025 the Key to Unlocking New Data Insights?

 The advent of AI has ushered in ground breaking changes in most areas of technology. AI is synonymous with a humongous amount of data, data...