Powershell CSV 到 SQL 查询答案

【问题标题】：Powershell CSV to SQL QueryPowershell CSV 到 SQL 查询
【发布时间】：2016-08-16 17:54:18
【问题描述】：

我回到了我对 powershell 的持续学习中，我已经达到了我的能力极限。所以想知道是否有人可以将我引向正确的方向，或者告诉我我想做的事情是否可行？

我编写了一个脚本，它从网络上下载许多文件，然后删除除我希望使用的数据之外的所有文件。这些是 .csv 的...

我已经添加了迄今为止我设法拼凑在一起的 Powershell 代码，下载数据时请参见下文。其中一个文件的数据提取如下所示。

A 列包含说明

B 列包含一个 ISIN（这是工作信息）

C 列包含一个数字

D 列是我希望将 SQL 查询数据返回到的位置

请找到示例数据的链接 Data Example Data Example

然后我想要实现的是根据 B 列中的数据运行以下 SQL 命令

Select *
From CL
Where CLISIN in ('GB0004835483',
'BE0003793107',
'GB00B7V2GY97',
'GB0000595859',
'GB00B1VCNQ84',
'GB0004992003',
'GB0002369352')

我相信我可能需要将其导出为另一个文件？虽然使用导出的最终结果，但我需要将它们放在此 csv 文件的 D 列中。

希望我已经说明了这一点，如果没有，请告诉我，我会尽可能广泛。

简而言之，我可以使用 powershell 自动为 B 列中的每个项目运行查询，并将该匹配查询的结果添加到 D 列中的相关行吗？

我无法通过谷歌或这里找到答案......

#### DOWNLOAD LOCATIONS ####
$DownloadPTMLocation =  "L:\Operations Database\TakeOverPanel\PTMDisclosureTable.xls"
$DownloadPTMCSVLocation =  "L:\Operations Database\TakeOverPanel\PTMDisclosureTable.csv"
$DownloadIPTMLocation = "L:\Operations Database\TakeOverPanel\IPTMDisclosureTable.xls"
$DownloadIPTMCSVLocation = "L:\Operations Database\TakeOverPanel\IPTMDisclosureTable.csv"

#### WEB URLS ###
$PTMURL = "http://www.thetakeoverpanel.org.uk/new/disclosureTable/v3/disclosuretable.xls"
$PTMCSVURL = "http://www.thetakeoverpanel.org.uk/new/disclosureTable/v3/disclosuretable.csv"
$IPTMURL = "http://irishtakeoverpanel.ie/disclosure/disclosuretable.xls"
$IPTMCSVURL = "http://irishtakeoverpanel.ie/disclosure/disclosuretable.csv"

$Path = Get-Location

#### Load Web Service #### 
$WebClient = New-Object System.Net.WebClient

#### Start download Process ####
Write-Host "Downloading PTM File 1 of 4" $Path -ForegroundColor Green
    $Url = $PTMURL
    $Path = $DownloadPTMLocation
    $WebClient.DownloadFile($PTMURL, $DownloadPTMLocation)

Write-Host "Downloading IPTM File 2 of 3" $Path -ForegroundColor Blue
    $Url = $IPTMURL 
    $Path = $DownloadIPTMLocation
    $WebClient.DownloadFile($IPTMURL, $DownloadIPTMLocation)

Write-Host "Downloading PTM Csv File 3 of 4" $Path -ForegroundColor Gray
    $Url = $PTMCSVURL
    $Path = $DownloadPTMCSVLocation
    $WebClient.DownloadFile($PTMCSVURL, $DownloadPTMCSVLocation)

Write-Host "Downloading IPTM File 4 of 4" $Path -ForegroundColor Red
    $Url = $IPTMCSVURL 
    $Path = $DownloadIPTMCSVLocation
    $WebClient.DownloadFile($IPTMCSVURL, $DownloadIPTMCSVLocation)



    #####################


    ## PTM ## 

#Customise Vars
$DownloadPTMCSVLocation = "L:\Operations Database\TakeOverPanel\PTMDisclosureTable.csv"
$OutputPTMCSVLocation = "L:\Operations Database\TakeOverPanel\PTMDisclosureTableb.csv"
$Match = "ISIN"
$Matchs = "NSI"

## Strips all lines that do not contain ISIN ##

(Get-Content $DownloadPTMCSVLocation) -match $Match | Out-File $OutputPTMCSVLocation

Remove-Item $DownloadPTMCSVLocation

Rename-Item $OutputPTMCSVLocation -NewName $DownloadPTMCSVLocation

(Get-Content $DownloadPTMCSVLocation) -match $Matchs | Out-File $OutputPTMCSVLocation

Remove-Item $DownloadPTMCSVLocation

Rename-Item $OutputPTMCSVLocation -NewName $DownloadPTMCSVLocation

Get-Content $DownloadPTMCSVLocation | % {
    $_ -replace 'ISIN: ',''
    } | % {
    $_ -replace 'NSI: ',''
    } | Set-Content $OutputPTMCSVLocation

因此，在这方面进一步工作并碰壁（再次返回 SQL 查询）这是迄今为止的工作代码......

###########
## BEGIN ##
###########

#### DOWNLOAD LOCATIONS ####

$DownloadPTMCSVLocation =  "L:\Operations Database\TakeOverPanel\PTMDisclosureTable.csv"
$OutputPTMCSVLocation = "L:\Operations Database\TakeOverPanel\PTMDisclosureTableb.csv"


#### WEB URLS ###

$PTMCSVURL = "http://www.thetakeoverpanel.org.uk/new/disclosureTable/v3/disclosuretable.csv"


$Path = Get-Location

#### Load Web Service #### 
$WebClient = New-Object System.Net.WebClient

#### Start download Process ####


Write-Host "Downloading PTM Csv" $Path -ForegroundColor Gray
    $Url = $PTMCSVURL
    $Path = $DownloadPTMCSVLocation
    $WebClient.DownloadFile($PTMCSVURL, $DownloadPTMCSVLocation)



#####################


## PTM ## 

$Match = "ISIN"
$Matchs = "NSI"

Import-Csv $DownloadPTMCSVLocation -Header @("A", "ISIN", "NSI", "Output") | #Import the CSV
  Where { $_.ISIN -like "ISIN: ????????????" -and $_.NSI -like "NSI:*" } | #Filter rows
  Foreach-Object {
    $_.ISIN = $_.ISIN.Replace("ISIN: ", "")
    $_.NSI = $_.NSI.Replace("NSI: ", "")

    $query = "select CLIALPHASORTCODE, vl2securitynum, sum(cast(vl2beneficial as float)) as beneficial 
    from t5vaultsl2 vl2 left outer join t5client cli
    on vl2.VL2CLIENTNUM = cli.CLICODE

    where vl2.vl2securitynum = '$($_.ISIN)'

    group by CLIALPHASORTCODE, VL2SECURITYNUM"


##Credentials## 

$MISA = 'xx.xx.x.xx'
$MISB = 'xx.xx.x.xx'
$userName = 'UN'
$PassWord='PW'
$DB = 'reporting'


## CREATE MIS CREDENTIALS ##
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Data Source=$MISA;Initial Catalog=$DB;
Initial Catalog=$DB;User ID=$userName;Password=$PassWord;"

## - Runs Script from Set Location

$SqlCmd = New-Object System.Data.SqlClient.SqlCommand;
$SqlCMD.CommandText = $query;
$SqlCmd.Connection = $SqlConnection;

所以脚本现在如下...

#### DOWNLOAD LOCATIONS ####

$DownloadPTMCSVLocation =  "L:\Operations Database\TakeOverPanel\PTMDisclosureTable.csv"
$OutputPTMCSVLocation = "L:\Operations Database\TakeOverPanel\PTMDisclosureTableb.csv"
$ExportLocation = "L:\Operations Database\TakeOverPanel\test.csv"
$ExportLocationb = "L:\Operations Database\TakeOverPanel\test.xml"


#### WEB URLS ###

$PTMCSVURL = "http://www.thetakeoverpanel.org.uk/new/disclosureTable/v3/disclosuretable.csv"
$Path = Get-Location


#### Load Web Service #### 
$WebClient = New-Object System.Net.WebClient

#### Start download Process ####


Write-Host "Downloading PTM Csv" $Path -ForegroundColor Gray
    $Url = $PTMCSVURL
    $Path = $DownloadPTMCSVLocation
    $WebClient.DownloadFile($PTMCSVURL, $DownloadPTMCSVLocation)



################
#### Query  ####
################

$query = @"
    select CLIALPHASORTCODE, vl2securitynum, sum(cast(vl2beneficial as float)) as beneficial 
    from t5vaultsl2 vl2 left outer join t5client cli
    on vl2.VL2CLIENTNUM = cli.CLICODE

    where vl2.vl2securitynum = '$($_.ISIN)'

    group by CLIALPHASORTCODE, VL2SECURITYNUM
"@;


#####################################

$Match = "ISIN"
$Matchs = "NSI"

## Prepare SQL ##

$MISA = 'xx.xx.x.xx'
$MISB = 'xx.xx.x.xx'
$userName = 'UN'
$PassWord='PW'
$DB = 'reporting'


## CREATE SQL Connection ##
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Data Source=$MISA;Initial Catalog=$DB;Initial Catalog=$DB;User ID=$userName;Password=$PassWord;"
$SqlConnection.Open()

# Put everything in a Try block so if there is an error the SQL
# connection is still closed
try
{
    $SqlCmd = $SqlConnection.CreateCommand()

    ## Process CSV ##

    Import-Csv $DownloadPTMCSVLocation -Header @("A", "ISIN", "NSI", "Output") | #Import the CSV
      Where { $_.ISIN -like "ISIN: ????????????" -and $_.NSI -like "NSI:*" } | #Filter rows
      Foreach-Object {
        $_.ISIN = $_.ISIN.Replace("ISIN: ", "")
        $_.NSI = $_.NSI.Replace("NSI: ", "")

        # Get data from SQL
        $query = "select CLIALPHASORTCODE, vl2securitynum, sum(cast(vl2beneficial as float)) as beneficial from t5vaultsl2 vl2 left outer join t5client cli on vl2.VL2CLIENTNUM = cli.CLICODE where vl2.vl2securitynum = '$($_.ISIN)' group by CLIALPHASORTCODE, VL2SECURITYNUM"

        $SqlCmd.CommandText  = $query
        $result = $SqlCmd.ExecuteReader()
        $table = New-Object "System.Data.DataTable"
        $table.Load($result)

        $t = $table.Vl2Beneficial

        # Pass row on through the pipeline
        $_.Output = $table.VL2Beneficial

        $table | Export-Clixml $ExportLocationb

        Write-Output $_
    } | Export-Csv $OutputPTMCSVLocation -NoTypeInformation
}
finally
{
    # Always close SQL connection even if error is encountered.
    $SqlConnection.Close()
}

【问题讨论】：

请提供您 CSV 文件中的数据（不是屏幕截图）。以及您想从该 CSV 获得什么输出。如果此 CSV 具有确定的结构，您可以使用 OPENROWSET 从 SQL Server 端的此文件中读取数据。
PTM 文件可以从提供的链接下载，应该足够了
嘿 Deadly_Bagel，是的，它们可以下载，并且在我的代码中我正在这样做，然后我正在做的是将我的资产与这些 PTM 文件进行比较。这就是 SQL 要求的来源。Gofr1，现在已经添加了所需的更改，我相信......我希望在 D 列中获得的输出是我的数字位置。
如果我理解正确，您需要根据您的示例使用您正在生成的电子表格的 B 列生成一个 SQL 查询，然后将所述 SQL 查询的每个相应结果放入 D 列？
Deadly-Bagel，确切地说......我可以想到其他冗长的方法，但如果我能提供帮助，我真的希望将所有内容包含在 powershell 中。

标签： sql-server loops csv powershell

【解决方案1】：

## PTM ## 

$Match = "ISIN"
$Matchs = "NSI"

## Prepare SQL ##

$MISA = 'xx.xx.x.xx'
$MISB = 'xx.xx.x.xx'
$userName = 'UN'
$PassWord='PW'
$DB = 'reporting'

## CREATE SQL Connection ##
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Data Source=$MISA;Initial Catalog=$DB;Initial Catalog=$DB;User ID=$userName;Password=$PassWord;"
$SqlConnection.Open()

# Put everything in a Try block so if there is an error the SQL
# connection is still closed
try
{
    $SqlCmd = $SqlConnection.CreateCommand()

    ## Process CSV ##

    Import-Csv $DownloadPTMCSVLocation -Header @("A", "ISIN", "NSI", "Output") | #Import the CSV
      Where { $_.ISIN -like "ISIN: ????????????" -and $_.NSI -like "NSI:*" } | #Filter rows
      Foreach-Object {
        $_.ISIN = $_.ISIN.Replace("ISIN: ", "")
        $_.NSI = $_.NSI.Replace("NSI: ", "")

        # Get data from SQL
        $query = "select CLIALPHASORTCODE, vl2securitynum, sum(cast(vl2beneficial as float)) as beneficial from t5vaultsl2 vl2 left outer join t5client cli on vl2.VL2CLIENTNUM = cli.CLICODE where vl2.vl2securitynum = '$($_.ISIN)' group by CLIALPHASORTCODE, VL2SECURITYNUM"

        $SqlCmd.CommandText  = $query
        $result = $SqlCmd.ExecuteReader()
        $table = New-Object "System.Data.DataTable"
        $table.Load($result)

        # Pass row on through the pipeline
        $_.Output = $table.VL2Beneficial

        Write-Output $_
    } | Export-Csv $OutputPTMCSVLocation -NoTypeInformation
}
finally
{
    # Always close SQL connection even if error is encountered.
    $SqlConnection.Close()
}

基本上我放弃了所有文件重命名，因为它完全没有必要（如果需要，可以使用变量来处理这类事情），Import-Csv 返回一个对象数组，每一行一个对象，所以我将它们传递给过滤器（其中 ISIN 被检查为仅 12 个字符）然后循环整理数据。

【讨论】：

在聊天中给你丢东西...进一步尝试让 SQL 输出并返回到我极度缺乏知识让我感到沮丧...我可以让它在 D 列中不返回任何内容，或 SQL 查询但不是 sql 查询的结果...
最后一个元素，在导出时看起来是一个错误， System.Object[] 被返回到结果中。有什么想法...如果需要，我可以发送完整的代码更新？
不，我想我知道发生了什么。见聊天。