【发布时间】:2019-07-08 12:28:41
【问题描述】:
我有使用 Azure 数据工厂在 Azure SQL 数据库上提取数据、处理和抽取数据的过程。最后,来自 Azure SQL 的数据由 SSAS 表格模型使用。到目前为止,直到将数据泵入 Azure SQL 的整个过程都是使用 Azure 数据工厂 v2 自动完成的,但我无法找到自动处理 SSAS 表格模型的方法。请让我知道是否有办法做到这一点。
我希望它由 ADF v2 以某种方式触发,以便表格在新数据被注入 SQL 时立即获取更新数据。由于我无法使用 ADF v2 做到这一点,我尝试使用 Powershell。我在这里面临的障碍是,我需要通过两个凭据来处理表格 1. Azure SQL 的 SQL 服务器身份验证 2. 模拟一个用户的 Windows 凭据 我对 Powershell 非常陌生,在传递这两个凭据时遇到问题
Powershell 脚本:-
param($ServerName="Your Server Name", $DBName="Your DB Name", $ProcessTypeDim="ProcessFull",$ProcessTypeMG="ProcessFull",
$Transactional="Y", $Parallel="Y",$MaxParallel=2,$MaxCmdPerBatch=1, $PrintCmd="N",
$logFilePath="Your Path where you want to save the log file")
## Add the AMO namespace
Write-Output "Process Starts for database $DBName :" | Out-File -FilePath $logFilePath
$loadInfo = [Reflection.Assembly]::LoadWithPartialName("Microsoft.AnalysisServices")
if ($Transactional -eq "Y") {$TransactionalB=$true} else {$TransactionalB=$false}
if ($Parallel -eq "Y") {$ParallelB=$true} else {$ParallelB=$false}
$server = New-Object Microsoft.AnalysisServices.Server
$server.connect($ServerName)
if ($server.name -eq $null) {
Write-Output ("Server '{0}' not found" -f $ServerName)
Write-Output "Server '{0}' not found" -f $ServerName | Out-File -FilePath $logFilePath -Append
break
}
$DB = $server.Databases.FindByName($DBName)
if ($DB -eq $null) {
Write-Output ("Database '{0}' not found" -f $DBName)
Write-Output "Database '{0}' not found" -f $DBName | Out-File -FilePath $logFilePath -Append
break
}
$date_Start=Get-Date
Write-Output("Load start time {0}" -f (Get-Date -uformat "%H:%M:%S") )
Write-Output "Load start time: " (Get-Date -uformat "%H:%M:%S") | Out-File -FilePath $logFilePath -Append
Write-Output("----------------------------------------------------------------")
Write-Output "----------------------------------------------------------------"| Out-File -FilePath $logFilePath -Append
Write-Output("Server : {0}" -f $Server.Name)
Write-Output "Server :" $Server.Name | Out-File -FilePath $logFilePath -Append
Write-Output("Database: {0}" -f $DB.Name)
Write-Output "Database: " $DB.Name | Out-File -FilePath $logFilePath -Append
Write-Output("DB State: {0}" -f $DB.State)
Write-Output "DB State: " $DB.State | Out-File -FilePath $logFilePath -Append
Write-Output("DB Size : {0}MB" -f
($DB.EstimatedSize/1024/1024).ToString("#,##0"))
Write-Output "DB Size : " ($DB.EstimatedSize/1024/1024).ToString("#,##0")| Out-File -FilePath $logFilePath -Append
Write-Output("----------------------------------------------------------------")
Write-Output "----------------------------------------------------------------"| Out-File -FilePath $logFilePath -Append
Write-Output("DB processing started. Time: {0}" -f (Get-Date -uformat "%H:%M:%S"))
Write-Output "DB processing started. Time: " (Get-Date -uformat "%H:%M:%S")| Out-File -FilePath $logFilePath -Append
$server.CaptureXml=$TRUE # Just capture server statements, dont execute them
#Process dimensions
foreach ($dim in $DB.Dimensions) {
$dim.Process($ProcessTypeDim)
} # Dimensions
#Process cubes
foreach ($cube in $DB.Cubes) {
foreach ($mg in $cube.MeasureGroups) {
foreach ($part in $mg.Partitions) {
$part.Process($ProcessTypeMG)
}
}
}
# Separate step to process all linked measure groups. Linke MG does not have partitions
foreach ($cube in $DB.Cubes) {
foreach ($mg in $cube.MeasureGroups) {
if ($mg.IsLinked) {
$mg.Process($ProcessTypeMG)
}
}
}
$server.CaptureXML = $FALSE # Finish capturing statements. All statements are in Server.CaptureLog
$cmdBatch = @"
<Batch xmlns="http://schemas.microsoft.com/analysisservices/2003/engine">
<Parallel MaxParallel="##MaxParallel##">
##ProcessCmd##
</Parallel>
</Batch>
"@
$cmdBatch = $cmdBatch -replace("##MaxParallel##",$MaxParallel)
#$ErrorActionPreference = "SilentlyContinue"
$currentCmdNo=0; $currentCmdInBatchNo=0;
$processCmd="";$currentBatchNo=0
$TotalCmdCount = $Server.CaptureLog.Count
foreach ($cmdLine in $Server.CaptureLog) {
$currentCmdNo = $currentCmdNo + 1
$processCmd = $processCmd + $cmdLine + "`n"
$currentCmdInBatchNo=$currentCmdInBatchNo + 1
if ($currentCmdInBatchNo -ge $MaxCmdPerBatch -or $currentCmdNo -eq $TotalCmdCount) { #MaxCmdPerBatch reached, execute commands
$processCmd = $cmdBatch -replace("##ProcessCmd##", $processCmd)
if ($PrintCmd -eq "Y") { Write-Output($processCmd) }
$currentBatchNo = $currentBatchNo + 1;
Write-Output("=== Startining batch No {0}. Time: {1} ..." -f $currentBatchNo, (Get-Date -uformat "%H:%M:%S"))
Write-Output "=== Startining batch No $currentBatchNo. Time: ..." (Get-Date -uformat "%H:%M:%S")| Out-File -FilePath $logFilePath -Append
$Result = $Server.Execute($processCmd)
# Report errors and warnings
foreach ($res in $Result) {
foreach ($msg in $res.Messages) {
if ($msg.Description -ne $null) {
Write-Output("{0}" -f $msg.Description)
Write-Output $msg.Description| Out-File -FilePath $logFilePath -Append
}
}
}
# Reset temp values
$processCmd = ""; $currentCmdInBatchNo=0;
}
}#foreach
Write-Output("-------------------------Cube status----------------------------")
Write-Output("-------------------------Cube status----------------------------")| Out-File -FilePath $logFilePath -Append
$data=$DB.Cubes|select name,state,lastprocessed
Write-Output($data)
Write-Output $data| Out-File -FilePath $logFilePath -Append
Write-Output("--------------------Dimension status----------------------------")
Write-Output("--------------------Dimension status----------------------------")| Out-File -FilePath $logFilePath -Append
$data=$DB.Dimensions|select name,state,lastprocessed
Write-Output($data)
Write-Output $data| Out-File -FilePath $logFilePath -Append
Write-Output("-----------------Dimension related to cubes---------------------")
Write-Output("-----------------Dimension related to cubes---------------------")| Out-File -FilePath $logFilePath -Append
foreach ($cube in $DB.Cubes) {
Write-Output("Cube Name: $cube")
Write-Output "Cube Name: $cube"| Out-File -FilePath $logFilePath -Append
foreach ($dim in $DB.Dimensions) {
Write-Output(" $dim")
Write-Output " $dim"| Out-File -FilePath $logFilePath -Append
}
}
Write-Output("----------------------------------------------------------------")
Write-Output("----------------------------------------------------------------")| Out-File -FilePath $logFilePath -Append
$date_End=Get-Date
Write-Output("Load End Time: {0}" -f (Get-Date -uformat "%H:%M:%S"))
Write-Output "Load End Time: " (Get-Date -uformat "%H:%M:%S")| Out-File -FilePath $logFilePath -Append
$ptime="Total Processing Time :"+($date_End-$date_Start).Hours+" Hours, "+($date_End-$date_Start).Minutes+" Mins, "+($date_End-$date_Start).Seconds+" Secs "
Write-Output $ptime
Write-Output $ptime | Out-File -FilePath $logFilePath -Append
错误信息 -
OLE DB 或 ODBC 错误:登录超时已过期; HYT00;网络相关 或在建立一个特定于实例的错误 连接到 SQL Server。服务器未找到或无法访问。查看 如果实例名称正确且 SQL Server 配置为允许 远程连接。有关详细信息,请参阅 SQL Server 联机丛书。 08001;命名管道提供程序:无法打开到 SQL Server 的连接 [5]。 ; 08001.
【问题讨论】:
-
您是否使用安装在主机上的 Azure 分析服务或 SSAS?如果您使用 Azure 分析服务,那么您可以使用 ADF2 中的 REST API 并处理您的模型。
-
我正在使用本地 SSAS
-
我以前做过,但它是通过 SSIS 包完成的。仅通过 Powershell 就可以实现。您是否在 Azure 自动化中运行 powershell?你得到的错误是网络,而不是登录。
-
到目前为止,我正在尝试在托管 SSAS 的同一台机器上运行 Powershell。我在本地机器上遇到了上述错误。
-
我猜它发生在
$server.connect($ServerName)。您输入了正确的服务器名称对吗? (作为参数传入)。一个很好的检查方法是把$ServerName放在前面的那一行,它会回显它。
标签: azure powershell automation ssas