[sql] How Stuff and 'For Xml Path' work in SQL Server?

Table is:

Id Name
1 aaa
1 bbb
1 ccc
1 ddd
1 eee

Required output:

Id abc
1 aaa,bbb,ccc,ddd,eee

Query:

SELECT ID, 
    abc = STUFF(
                 (SELECT ',' + name FROM temp1 FOR XML PATH ('')), 1, 1, ''
               ) 
FROM temp1 GROUP BY id

This query is working properly. But I just need the explanation how it works or is there any other or short way to do this.

I am getting very confused to understand this.

This question is related to sql sql-server for-xml-path stuff

The answer is


Here is how it works:

1. Get XML element string with FOR XML

Adding FOR XML PATH to the end of a query allows you to output the results of the query as XML elements, with the element name contained in the PATH argument. For example, if we were to run the following statement:

SELECT ',' + name 
              FROM temp1
              FOR XML PATH ('')

By passing in a blank string (FOR XML PATH('')), we get the following instead:

,aaa,bbb,ccc,ddd,eee

2. Remove leading comma with STUFF

The STUFF statement literally "stuffs” one string into another, replacing characters within the first string. We, however, are using it simply to remove the first character of the resultant list of values.

SELECT abc = STUFF((
            SELECT ',' + NAME
            FROM temp1
            FOR XML PATH('')
            ), 1, 1, '')
FROM temp1

The parameters of STUFF are:

  • The string to be “stuffed” (in our case the full list of name with a leading comma)
  • The location to start deleting and inserting characters (1, we’re stuffing into a blank string)
  • The number of characters to delete (1, being the leading comma)

So we end up with:

aaa,bbb,ccc,ddd,eee

3. Join on id to get full list

Next we just join this on the list of id in the temp table, to get a list of IDs with name:

SELECT ID,  abc = STUFF(
             (SELECT ',' + name 
              FROM temp1 t1
              WHERE t1.id = t2.id
              FOR XML PATH (''))
             , 1, 1, '') from temp1 t2
group by id;

And we have our result:

Id Name
1 aaa,bbb,ccc,ddd,eee

Hope this helps!


This article covers various ways of concatenating strings in SQL, including an improved version of your code which doesn't XML-encode the concatenated values.

SELECT ID, abc = STUFF
(
    (
        SELECT ',' + name
        FROM temp1 As T2
        -- You only want to combine rows for a single ID here:
        WHERE T2.ID = T1.ID
        ORDER BY name
        FOR XML PATH (''), TYPE
    ).value('.', 'varchar(max)')
, 1, 1, '')
FROM temp1 As T1
GROUP BY id

To understand what's happening, start with the inner query:

SELECT ',' + name
FROM temp1 As T2
WHERE T2.ID = 42 -- Pick a random ID from the table
ORDER BY name
FOR XML PATH (''), TYPE

Because you're specifying FOR XML, you'll get a single row containing an XML fragment representing all of the rows.

Because you haven't specified a column alias for the first column, each row would be wrapped in an XML element with the name specified in brackets after the FOR XML PATH. For example, if you had FOR XML PATH ('X'), you'd get an XML document that looked like:

<X>,aaa</X>
<X>,bbb</X>
...

But, since you haven't specified an element name, you just get a list of values:

,aaa,bbb,...

The .value('.', 'varchar(max)') simply retrieves the value from the resulting XML fragment, without XML-encoding any "special" characters. You now have a string that looks like:

',aaa,bbb,...'

The STUFF function then removes the leading comma, giving you a final result that looks like:

'aaa,bbb,...'

It looks quite confusing at first glance, but it does tend to perform quite well compared to some of the other options.


PATH mode is used in generating XML from a SELECT query

1. SELECT   
       ID,  
       Name  
FROM temp1
FOR XML PATH;  

Ouput:
<row>
<ID>1</ID>
<Name>aaa</Name>
</row>

<row>
<ID>1</ID>
<Name>bbb</Name>
</row>

<row>
<ID>1</ID>
<Name>ccc</Name>
</row>

<row>
<ID>1</ID>
<Name>ddd</Name>
</row>

<row>
<ID>1</ID>
<Name>eee</Name>
</row>

The Output is element-centric XML where each column value in the resulting rowset is wrapped in an row element. Because the SELECT clause does not specify any aliases for the column names, the child element names generated are the same as the corresponding column names in the SELECT clause.

For each row in the rowset a tag is added.

2.
SELECT   
       ID,  
       Name  
FROM temp1
FOR XML PATH('');

Ouput:
<ID>1</ID>
<Name>aaa</Name>
<ID>1</ID>
<Name>bbb</Name>
<ID>1</ID>
<Name>ccc</Name>
<ID>1</ID>
<Name>ddd</Name>
<ID>1</ID>
<Name>eee</Name>

For Step 2: If you specify a zero-length string, the wrapping element is not produced.

3. 

    SELECT   

           Name  
    FROM temp1
    FOR XML PATH('');

    Ouput:
    <Name>aaa</Name>
    <Name>bbb</Name>
    <Name>ccc</Name>
    <Name>ddd</Name>
    <Name>eee</Name>

4. SELECT   
        ',' +Name  
FROM temp1
FOR XML PATH('')

Ouput:
,aaa,bbb,ccc,ddd,eee

In Step 4 we are concatenating the values.

5. SELECT ID,
    abc = (SELECT   
            ',' +Name  
    FROM temp1
    FOR XML PATH('') )
FROM temp1

Ouput:
1   ,aaa,bbb,ccc,ddd,eee
1   ,aaa,bbb,ccc,ddd,eee
1   ,aaa,bbb,ccc,ddd,eee
1   ,aaa,bbb,ccc,ddd,eee
1   ,aaa,bbb,ccc,ddd,eee


6. SELECT ID,
    abc = (SELECT   
            ',' +Name  
    FROM temp1
    FOR XML PATH('') )
FROM temp1 GROUP by iD

Ouput:
ID  abc
1   ,aaa,bbb,ccc,ddd,eee

In Step 6 we are grouping the date by ID.

STUFF( source_string, start, length, add_string ) Parameters or Arguments source_string The source string to modify. start The position in the source_string to delete length characters and then insert add_string. length The number of characters to delete from source_string. add_string The sequence of characters to insert into the source_string at the start position.

SELECT ID,
    abc = 
    STUFF (
        (SELECT   
                ',' +Name  
        FROM temp1
        FOR XML PATH('')), 1, 1, ''
    )
FROM temp1 GROUP by iD

Output:
-----------------------------------
| Id        | Name                |
|---------------------------------|
| 1         | aaa,bbb,ccc,ddd,eee |
-----------------------------------

There is very new functionality in Azure SQL Database and SQL Server (starting with 2017) to handle this exact scenario. I believe this would serve as a native official method for what you are trying to accomplish with the XML/STUFF method. Example:

select id, STRING_AGG(name, ',') as abc
from temp1
group by id

STRING_AGG - https://msdn.microsoft.com/en-us/library/mt790580.aspx

EDIT: When I originally posted this I made mention of SQL Server 2016 as I thought I saw that on a potential feature that was to be included. Either I remembered that incorrectly or something changed, thanks for the suggested edit fixing the version. Also, pretty impressed and wasn't fully aware of the multi-step review process that just pulled me in for a final option.


In for xml path, if we define any value like [ for xml path('ENVLOPE') ] then these tags will be added with each row:

<ENVLOPE>
</ENVLOPE>

SELECT ID, 
    abc = STUFF(
                 (SELECT ',' + name FROM temp1 FOR XML PATH ('')), 1, 1, ''
               ) 
FROM temp1 GROUP BY id

Here in the above query STUFF function is used to just remove the first comma (,) from the generated xml string (,aaa,bbb,ccc,ddd,eee) then it will become (aaa,bbb,ccc,ddd,eee).

And FOR XML PATH('') simply converts column data into (,aaa,bbb,ccc,ddd,eee) string but in PATH we are passing '' so it will not create a XML tag.

And at the end we have grouped records using ID column.


I did debugging and finally returned my 'stuffed' query to it it's normal way.

Simply

select * from myTable for xml path('myTable')

gives me contents of the table to write to a log table from a trigger I debug.


Declare @Temp As Table (Id Int,Name Varchar(100))
Insert Into @Temp values(1,'A'),(1,'B'),(1,'C'),(2,'D'),(2,'E'),(3,'F'),(3,'G'),(3,'H'),(4,'I'),(5,'J'),(5,'K')
Select X.ID,
stuff((Select ','+ Z.Name from @Temp Z Where X.Id =Z.Id For XML Path('')),1,1,'')
from @Temp X
Group by X.ID

STUFF((SELECT distinct ',' + CAST(T.ID) FROM Table T where T.ID= 1 FOR XML PATH('')),1,1,'') AS Name


Questions with sql tag:

Passing multiple values for same variable in stored procedure SQL permissions for roles Generic XSLT Search and Replace template Access And/Or exclusions Pyspark: Filter dataframe based on multiple conditions Subtracting 1 day from a timestamp date PYODBC--Data source name not found and no default driver specified select rows in sql with latest date for each ID repeated multiple times ALTER TABLE DROP COLUMN failed because one or more objects access this column Create Local SQL Server database Export result set on Dbeaver to CSV How to create temp table using Create statement in SQL Server? SQL Query Where Date = Today Minus 7 Days How do I pass a list as a parameter in a stored procedure? #1273 – Unknown collation: ‘utf8mb4_unicode_520_ci’ MySQL Error: : 'Access denied for user 'root'@'localhost' SQL Server IF EXISTS THEN 1 ELSE 2 How to add a boolean datatype column to an existing table in sql? Presto SQL - Converting a date string to date format What is the meaning of <> in mysql query? Change Date Format(DD/MM/YYYY) in SQL SELECT Statement Convert timestamp to date in Oracle SQL #1292 - Incorrect date value: '0000-00-00' Postgresql tables exists, but getting "relation does not exist" when querying SQL query to check if a name begins and ends with a vowel Find the number of employees in each department - SQL Oracle Error in MySQL when setting default value for DATE or DATETIME Drop view if exists Could not find server 'server name' in sys.servers. SQL Server 2014 How to create a Date in SQL Server given the Day, Month and Year as Integers TypeError: tuple indices must be integers, not str Select Rows with id having even number SELECT list is not in GROUP BY clause and contains nonaggregated column IN vs ANY operator in PostgreSQL How to insert date values into table Error related to only_full_group_by when executing a query in MySql How to select the first row of each group? Connecting to Microsoft SQL server using Python eloquent laravel: How to get a row count from a ->get() How to execute raw queries with Laravel 5.1? In Oracle SQL: How do you insert the current date + time into a table? Extract number from string with Oracle function Rebuild all indexes in a Database SQL: Two select statements in one query DB2 SQL error sqlcode=-104 sqlstate=42601 What difference between the DATE, TIME, DATETIME, and TIMESTAMP Types How to run .sql file in Oracle SQL developer tool to import database? Concatenate columns in Apache Spark DataFrame How Stuff and 'For Xml Path' work in SQL Server? Fatal error: Call to a member function query() on null

Questions with sql-server tag:

Passing multiple values for same variable in stored procedure SQL permissions for roles Count the Number of Tables in a SQL Server Database Visual Studio 2017 does not have Business Intelligence Integration Services/Projects ALTER TABLE DROP COLUMN failed because one or more objects access this column Create Local SQL Server database How to create temp table using Create statement in SQL Server? SQL Query Where Date = Today Minus 7 Days How do I pass a list as a parameter in a stored procedure? SQL Server date format yyyymmdd SQL Server IF EXISTS THEN 1 ELSE 2 'Microsoft.ACE.OLEDB.16.0' provider is not registered on the local machine. (System.Data) How to add a boolean datatype column to an existing table in sql? How to import an Excel file into SQL Server? How to use the COLLATE in a JOIN in SQL Server? Change Date Format(DD/MM/YYYY) in SQL SELECT Statement Stored procedure with default parameters Drop view if exists Violation of PRIMARY KEY constraint. Cannot insert duplicate key in object How to update large table with millions of rows in SQL Server? Could not find server 'server name' in sys.servers. SQL Server 2014 How to create a Date in SQL Server given the Day, Month and Year as Integers Select Rows with id having even number A connection was successfully established with the server, but then an error occurred during the login process. (Error Number: 233) SQL Server: Error converting data type nvarchar to numeric How to add LocalDB to Visual Studio 2015 Community's SQL Server Object Explorer? Using DISTINCT along with GROUP BY in SQL Server Rebuild all indexes in a Database How to generate Entity Relationship (ER) Diagram of a database using Microsoft SQL Server Management Studio? The target principal name is incorrect. Cannot generate SSPI context How Stuff and 'For Xml Path' work in SQL Server? How to view the roles and permissions granted to any database user in Azure SQL server instance? How do I create a local database inside of Microsoft SQL Server 2014? Format number as percent in MS SQL Server MSSQL Regular expression How to select all the columns of a table except one column? SQL count rows in a table EXEC sp_executesql with multiple parameters SQL Server : How to test if a string has only digit characters Conversion of a varchar data type to a datetime data type resulted in an out-of-range value in SQL query Remove decimal values using SQL query How to drop all tables from a database with one SQL query? How to get last 7 days data from current datetime to last 7 days in sql server Get last 30 day records from today date in SQL Server Using Excel VBA to run SQL query No process is on the other end of the pipe (SQL Server 2012) How to subtract 30 days from the current date using SQL Server Calculate time difference in minutes in SQL Server How to join two tables by multiple columns in SQL? The database cannot be opened because it is version 782. This server supports version 706 and earlier. A downgrade path is not supported

Questions with for-xml-path tag:

How Stuff and 'For Xml Path' work in SQL Server?

Questions with stuff tag:

How Stuff and 'For Xml Path' work in SQL Server?