SQL Server Exporting Data Programatically with bcp and xp_cmdshell


Many times you may find that you need to take some data in a table or tables and export that data, in a specific format, to an external text (flat) file, often for the purpose of sending that file to another computer system, perhaps at another department or company.

This post discusses, by example, how to do that. There is an article on the Web called Exporting Data Programatically with bcp and xp_cmdshell that discusses how this works. Here is another article called Working with the bcp Command-line Utility.

Admittedly the following code is not very simple and is a bit longer that I would like it to be, but in the real world your code will be likely be more complicated and longer than this code.

Below are three screen shots of the original Authors table contents, the contents of the intermediate table called AuthorsFormatted, and a screen shot of the data of the outputted file as seen in NotePad++ after the data is selected.



Below is the SQL Server query code used in this example. The AuthorsFormatted table was already created at an earlier time. It has one column called [Output] with a data type of char(20).

1USE myDatabase
2GO
3-- Purpose: To create a text output file from a table.
4-- Inputs: dbo.Authors table
5-- Outputs: a re-formatted text file
6--
7-- Comments:
8-- We need to create an intermediary table to hold the
9-- formatted data from dbo.Authors because we are using
10-- parameters. bcp and xp_cmdshell won't work otherwise.
11-- The formatted one-column table: dbo.AuthorsFormatted
12--
13-- The dbo.Authors table is one I created that only has 2
14-- records(rows) and 2 columns: [FirstName] and [LastName]
15--
16-- The output text file has a header, detail lines and a trailer.
17--
18DECLARE @sql varchar(8000)  -- for bcp
19DECLARE @sqlselect as varchar(8000)  -- for xp_cmdshell
20-- data used inside the output file:
21DECLARE @Type as char(1)
22DECLARE @Number as varchar(5)
23-- data used in the output directory and file name:
24DECLARE @datetoday as smalldatetime
25DECLARE @directorylocation as varchar(200) -- of output text file
26DECLARE @firstpartfilename as varchar(10)
27-- We will include the current date in the text file name
28-- We have the date (@datetoday) already
29DECLARE @YearString as varchar(4)   /* i.e. 2017 */
30DECLARE @MonthString as varchar(2)
31DECLARE @DayString as varchar(2)
32DECLARE @MonthChar as char(2)  -- will have leading zeros
33DECLARE @DayChar as char(2)  -- will have leading zeros
34--
35SET @Type = 'C'  -- just a hard-coded constant
36SET @Number = 726  -- just a hard-coded number (will have leading zeros)
37SET @datetoday = getdate()
38SET @directorylocation = 'C:\data\temp stuff\'
39SET @firstpartfilename = 'authors'
40--
41SET @YearString = cast(DATEPART(yyyy, @datetoday) as varchar(4));
42SET @MonthString = cast(DATEPART(mm, @datetoday) as varchar(2));
43SET @DayString = cast(DATEPART(dd, @datetoday) as varchar(2));
44-- put leading zeros in fromt of month and day if the month
45-- and day are only one character long (it sorts better that way)
46SET @MonthChar = cast(right('00' + @MonthString,2) as char(2))
47SET @DayChar = cast(right('00' + @DayString,2) as char(2))
48---------------------------------------------------------------------------------
49DELETE FROM dbo.AuthorsFormatted
50INSERT INTO dbo.AuthorsFormatted
51    SELECT
52        cast('hello' as char(5))
53        + cast(' world' as char(6))
54        + cast (@Type as char(1)) AS OutputCol
55    UNION ALL
56    SELECT cast([firstname] as char(10))
57        + cast(' ' as char(1))
58        + cast([lastname] as char(20)) AS OutputCol
59        FROM dbo.Authors
60    UNION ALL
61    SELECT cast('T' as char(1))
62        + cast(right('00000' + @Number,5) as char(5))
63----------------------------------------------------------------------------------
64SELECT * FROM dbo.AuthorsFormatted -- just display results to the SQL Server user:
65----------------------------------------------------------------------------------
66SET @sqlselect = 'SELECT Output FROM myDatabase.dbo.AuthorsFormatted'
67SET @sql = 'bcp "' + @sqlselect
68        + '" queryout "'
69        + @directorylocation
70        + @firstpartfilename
71        + @YearString + @MonthChar + @DayChar
72        + '.txt" -c -T -S'
73        + @@servername
74PRINT N'sql string to be sent to xp_cmdshell to create output file: '
75PRINT @sql
76EXEC master..xp_cmdshell @sql

When the above script is run the results output in the SQL server query window is as follows. It will appear a little different in the query window itself because we are using word-wrap in this blog post on WordPress whereas no word-wrap is used in the SQL Server Results window.

1(4 row(s) affected)
2 
3(4 row(s) affected)
4Output
5--------------------
6hello worldC       
7John       Smith   
8Sally      Jackson 
9T00726             
10 
11(4 row(s) affected)
12 
13sql string to be sent to xp_cmdshell to create output file:
14bcp "SELECT Output FROM myDatabase.dbo.AuthorsFormatted" queryout "C:\data\temp stuff\authors20170228.txt" -c -T -SDESKTOP-M9A1T3K\SQLEXPRESS
15output
16---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
17NULL
18Starting copy...
19NULL
204 rows copied.
21Network packet size (bytes): 4096
22Clock Time (ms.) Total     : 15     Average : (266.67 rows per sec.)
23NULL
24 
25(7 row(s) affected)