-
Notifications
You must be signed in to change notification settings - Fork 375
Add OneLocBuild template to arcade #6977
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
0a69efb
48b918c
c092f29
e359d25
629583b
716f6e2
d55cfa5
4247a53
6fddabf
d371b54
b7d38a1
8d1c843
d7ce6e5
1b77d81
2166815
67b82fc
425e6a8
0e964c7
34dbbce
a11152c
f6e8783
d523d65
289b6ad
022b3cc
27f690e
c579126
82ce3d5
da6f3fb
c341691
a67067e
45625ab
e24b533
0ee488f
893204a
3dd525b
c08645c
17a77fe
2801ff3
1965668
ecb081e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,101 @@ | ||
Param( | ||
[Parameter(Mandatory=$true)][string] $SourcesDirectory, # Directory where source files live; if using a Localize directory it should live in here | ||
[string] $LanguageSet = 'VS_Main_Languages', # Language set to be used in the LocProject.json | ||
[switch] $UseCheckedInLocProjectJson, # When set, generates a LocProject.json and compares it to one that already exists in the repo; otherwise just generates one | ||
[switch] $CreateNeutralXlfs # Creates neutral xlf files. Only set to false when running locally | ||
) | ||
|
||
# Generates LocProject.json files for the OneLocBuild task. OneLocBuildTask is described here: | ||
# https://ceapex.visualstudio.com/CEINTL/_wiki/wikis/CEINTL.wiki/107/Localization-with-OneLocBuild-Task | ||
|
||
Set-StrictMode -Version 2.0 | ||
$ErrorActionPreference = "Stop" | ||
. $PSScriptRoot\tools.ps1 | ||
|
||
Import-Module -Name (Join-Path $PSScriptRoot 'native\CommonLibrary.psm1') | ||
|
||
$exclusionsFilePath = "$SourcesDirectory\Localize\LocExclusions.json" | ||
$exclusions = @{ Exclusions = @() } | ||
if (Test-Path -Path $exclusionsFilePath) | ||
{ | ||
$exclusions = Get-Content "$exclusionsFilePath" | ConvertFrom-Json | ||
} | ||
|
||
Push-Location "$SourcesDirectory" # push location for Resolve-Path -Relative to work | ||
|
||
# Template files | ||
$jsonFiles = @() | ||
$jsonFiles += Get-ChildItem -Recurse -Path "$SourcesDirectory" | Where-Object { $_.FullName -Match "\.template\.config\\localize\\en\..+\.json" } # .NET templating pattern | ||
$jsonFiles += Get-ChildItem -Recurse -Path "$SourcesDirectory" | Where-Object { $_.FullName -Match "en\\strings\.json" } # current winforms pattern | ||
|
||
$xlfFiles = @() | ||
|
||
$allXlfFiles = Get-ChildItem -Recurse -Path "$SourcesDirectory\*\*.xlf" | ||
$langXlfFiles = @() | ||
if ($allXlfFiles) { | ||
$null = $allXlfFiles[0].FullName -Match "\.([\w-]+)\.xlf" # matches '[langcode].xlf' | ||
$firstLangCode = $Matches.1 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What happens if there is more than one "langcode" in the repo (maybe for test assets or soemthing). It seems like the code should at least produce some sort of warning in that situation, since it's going to pick one arbitrarily at that point (and potentially not even deterministically, since we aren't sorting anything) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Turns out that it doesn't matter -- it just needs a file with a langcode since all xlfs also contain English text equivalents, so we just take the first one and make it the "generic" xlf. This solution is actually ripped directly from their docs: https://ceapex.visualstudio.com/CEINTL/_wiki/wikis/CEINTL.wiki/1450/OneLocBuild-Non-Enu-source-file-support-(workaround) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. But this implies that every set of "xlf" files in the entire repository have the same set of languages. What if I have This means that There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is that first sentence going to be something this code assumes, but doesn't validate? That seems... scary... And validating it shouldn't be too hard... just search for ALL xlf files, and then group them by the first part. (That'd be a lot easier if this was C# and not powershell) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Theoretically "dotnet script" is a thing. :-) Just sayin... There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm fairly certain that it's a valid assumption to make because the LanguageSet variable applies to the entire repository -- all xlfs will be translated into the same set of languages. I agree that the second solution is more robust but it's pretty difficult in powershell... that's exactly what I would have done had this been C# haha. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ouch, quite the assumption for the tool to make, especially with repositories as large as ours. I hope that doesn't bite us. It... seems to hold for the repos I happen to have on my box? :-) |
||
$langXlfFiles = Get-ChildItem -Recurse -Path "$SourcesDirectory\*\*.$firstLangCode.xlf" | ||
} | ||
$langXlfFiles | ForEach-Object { | ||
$null = $_.Name -Match "([^.]+)\.[\w-]+\.xlf" # matches '[filename].[langcode].xlf' | ||
|
||
$destinationFile = "$($_.Directory.FullName)\$($Matches.1).xlf" | ||
$xlfFiles += Copy-Item "$($_.FullName)" -Destination $destinationFile -PassThru | ||
} | ||
|
||
$locFiles = $jsonFiles + $xlfFiles | ||
|
||
$locJson = @{ | ||
Projects = @( | ||
@{ | ||
LanguageSet = $LanguageSet | ||
LocItems = @( | ||
$locFiles | ForEach-Object { | ||
$outputPath = "Localize\$(($_.DirectoryName | Resolve-Path -Relative) + "\")" | ||
$continue = $true | ||
foreach ($exclusion in $exclusions.Exclusions) { | ||
if ($outputPath.Contains($exclusion)) | ||
{ | ||
$continue = $false | ||
} | ||
} | ||
$sourceFile = ($_.FullName | Resolve-Path -Relative) | ||
if (!$CreateNeutralXlfs -and $_.Extension -eq '.xlf') { | ||
Remove-Item -Path $sourceFile | ||
} | ||
if ($continue) | ||
{ | ||
return @{ | ||
SourceFile = $sourceFile | ||
CopyOption = "LangIDOnName" | ||
OutputPath = $outputPath | ||
} | ||
} | ||
} | ||
) | ||
} | ||
) | ||
} | ||
|
||
$json = ConvertTo-Json $locJson -Depth 5 | ||
Write-Host "(NETCORE_ENGINEERING_TELEMETRY=Build) LocProject.json generated:`n`n$json`n`n" | ||
jonfortescue marked this conversation as resolved.
Show resolved
Hide resolved
|
||
Pop-Location | ||
|
||
if (!$UseCheckedInLocProjectJson) { | ||
New-Item "$SourcesDirectory\Localize\LocProject.json" -Force # Need this to make sure the Localize directory is created | ||
Set-Content "$SourcesDirectory\Localize\LocProject.json" $json | ||
} | ||
else { | ||
New-Item "$SourcesDirectory\Localize\LocProject-generated.json" -Force # Need this to make sure the Localize directory is created | ||
Set-Content "$SourcesDirectory\Localize\LocProject-generated.json" $json | ||
|
||
if ((Get-FileHash "$SourcesDirectory\Localize\LocProject-generated.json").Hash -ne (Get-FileHash "$SourcesDirectory\Localize\LocProject.json").Hash) { | ||
Write-PipelineTaskError -Type "warning" -Message "Existing LocProject.json differs from generated LocProject.json. Download LocProject-generated.json and compare them." | ||
|
||
exit 1 | ||
} | ||
else { | ||
Write-Host "Generated LocProject.json and current LocProject.json are identical." | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,75 @@ | ||
parameters: | ||
# Optional: dependencies of the job | ||
dependsOn: '' | ||
|
||
# Optional: A defined YAML pool - https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=vsts&tabs=schema#pool | ||
pool: | ||
vmImage: vs2017-win2016 | ||
|
||
CeapexPat: $(dn-bot-ceapex-package-r) # PAT for the loc AzDO instance https://dev.azure.com/ceapex | ||
GithubPat: $(BotAccount-dotnet-bot-repo-PAT) | ||
|
||
SourcesDirectory: $(Build.SourcesDirectory) | ||
CreatePr: true | ||
UseCheckedInLocProjectJson: false | ||
LanguageSet: VS_Main_Languages | ||
LclSource: lclFilesInRepo | ||
LclPackageId: '' | ||
RepoType: gitHub | ||
|
||
jobs: | ||
- job: OneLocBuild | ||
|
||
dependsOn: ${{ parameters.dependsOn }} | ||
|
||
displayName: OneLocBuild | ||
|
||
pool: ${{ parameters.pool }} | ||
|
||
variables: | ||
- group: OneLocBuildVariables # Contains the CeapexPat and GithubPat | ||
- name: _GenerateLocProjectArguments | ||
value: -SourcesDirectory ${{ parameters.SourcesDirectory }} | ||
-LanguageSet "${{ parameters.LanguageSet }}" | ||
-CreateNeutralXlfs | ||
- ${{ if eq(parameters.UseCheckedInLocProjectJson, 'true') }}: | ||
- name: _GenerateLocProjectArguments | ||
value: ${{ variables._GenerateLocProjectArguments }} -UseCheckedInLocProjectJson | ||
|
||
|
||
steps: | ||
- task: Powershell@2 | ||
jonfortescue marked this conversation as resolved.
Show resolved
Hide resolved
|
||
inputs: | ||
filePath: $(Build.SourcesDirectory)/eng/common/generate-locproject.ps1 | ||
arguments: $(_GenerateLocProjectArguments) | ||
displayName: Generate LocProject.json | ||
|
||
- task: OneLocBuild@2 | ||
displayName: OneLocBuild | ||
inputs: | ||
locProj: Localize/LocProject.json | ||
outDir: $(Build.ArtifactStagingDirectory) | ||
lclSource: ${{ parameters.LclSource }} | ||
lclPackageId: ${{ parameters.LclPackageId }} | ||
isCreatePrSelected: ${{ parameters.CreatePr }} | ||
repoType: ${{ parameters.RepoType }} | ||
gitHubPatVariable: "${{ parameters.GithubPat }}" | ||
packageSourceAuth: patAuth | ||
patVariable: ${{ parameters.CeapexPat }} | ||
condition: always() | ||
|
||
- task: PublishBuildArtifacts@1 | ||
displayName: Publish Localization Files | ||
inputs: | ||
PathtoPublish: '$(Build.ArtifactStagingDirectory)/loc' | ||
PublishLocation: Container | ||
ArtifactName: Loc | ||
condition: always() | ||
|
||
- task: PublishBuildArtifacts@1 | ||
displayName: Publish LocProject.json | ||
inputs: | ||
PathtoPublish: '$(Build.SourcesDirectory)/Localize/' | ||
PublishLocation: Container | ||
ArtifactName: Loc | ||
condition: always() |
Uh oh!
There was an error while loading. Please reload this page.