Whitespace can be relevant in extracted XLIFF translation files.
Some i18n tools - e.g. CAT tool (OmegaT) - will reformat
the file to collapse whitespace if there is no indication to tell it
not to.
This commit adds the ability to specify "format options" that are passed
to the translation file serializer. The XLIFF 1.2 and 2.0 seralizers have
been updated to accept `{"xml:space":"preserve"}` format option which will
by added to the `<file>` element in the serialized translation file during
extraction.
Fixes#38679
PR Close#38737
A recent change to `@angular/localize` brought in the `AbsoluteFsPath` type
from the `@angular/compiler-cli`. But this brought along with it a reference
to NodeJS typings - specifically the `FileSystem` interface refers to the
`Buffer` type from NodeJS.
This affects compilation of `@angular/localize` code that will be run in
the browser - for example projects that reference `loadTranslations()`.
The compilation breaks if the NodeJS typings are not included in the build.
Clearly it is not desirable to have these typings included when the project
is not targeting NodeJS.
This commit replaces references to the NodeJS `Buffer` type with `Uint8Array`,
which is available across all platforms and is actually the super-class of
`Buffer`.
Fixes#38692
PR Close#38700
Previously, the location of a translation message, in XLIFF 2, was only
rendered if there were also notes for meaning or description. Now the
location will be rendered even if the other metadata is not provided.
Fixes#38705
PR Close#38713
Previously this package was installed in the default `dependencies` section
of `package.json`, but this meant that its own dependencies are treated as
dependencies of the main project: Babel, for example.
Generally, $localize` is not used at runtime - it is compiled out by the
translation tooling, so there is no need for it to be a full dependency.
In fact, even if it is used at runtime, the package itself is only used
at dev-time since the runtime bits will be bundled into a distributable.
So putting this package in `devDependencies` would only prevent libraries
from bringing the package into application projects that used them. This
is probably good in itself, since it should be up to the downstream project
to decide if it wants to include `@angular/localize` at runtime.
This commit changes the default location of the package to be the
`devDependencies` section, but gives an option `useAtRuntime` to choose
otherwise.
Fixes#38329
PR Close#38680
Previously, the `startSourceSpan` property could be null
but in reality it is always well defined - except for a legacy
case in the old i18n extraction/merging code, where the
typings for source-spans are already being undermined.
Making this property non-null, simplifies code elsewhere
in the project.
PR Close#38581
Previously the position of the error in a translation file when parsing
it was not displayed. Just the error message.
Now the position (line and column) and some context is displayed
along with the error messages.
Fixes#38377
PR Close#38673
When extracting i18n messages from source code, the XLIFF
serializers were missing some required attributes on the `<file>`
element.
This commit re-introduces the `original` property to each of XLIFF 1.2
and 2.0 serializers. Also it adds in the required `id` property for the
XLIFF 2.0 seralizer.
Fixes#38570
PR Close#38575
Formats like XLIFF allow the text of the original source to
be included as metadata. This commit fixes the message
extractor to also render this text when available.
PR Close#38536
Some translation file formats would like to be able to render the
text of placeholders taken from the original source files. This commit
adds this information to the extracted messages so that it can be
used in translation file serializers.
PR Close#38536
Previously, exceptions that were not `BabelParseError`s were just ignored.
Such exceptions are most likely programming errors in the package.
They are now re-thrown to ensure that the error is not hidden.
PR Close#38536
This commit is a tidy up of the translate plugin unit tests, but also ensures
that the tests are run in the context of a mock FileSystem. This ensures
that the tests are resilient to future refactors of the plugins that will
require a FileSystem to be initialized.
PR Close#38536
When extracting messages, source-mapping information is used to find
the original location of the message being extracted. This commit will
now include the text from the original source in the message location
so that it can be serialized into the translation file.
PR Close#38536
In preparation for supporting `equiv-text` placeholder information in
extracted translation files, this commit adds these optional properties
to the `ParsedMessage` interface and updates `parseMessage()` to
be able to store them.
PR Close#38536
Previously, if `useLegacyIds` was enabled, the message extractor
was always rendering the legacy message ids in translation
files even if an explicit "custom message id" had been provided
in the original message.
PR Close#38498
Previously nested container placeholders (i.e. HTML elements) were
not being fully parsed from translation files. This resulted in bad
translation of messages that contain these placeholders.
Note that this causes the canonical message ID to change for
such messages. Currently all messages generated from
templates use "legacy" message ids that are not affected by
this change, so this fix should not be seen as a breaking change.
Fixes#38422
PR Close#38452
When creating a `ParsedTranslation` from a set of message parts and
placeholder names a textual representation of the message is computed.
Previously the last placeholder and text segment were missing from this
computed message string.
PR Close#38452
Previously placeholders were only rendered for dynamic interpolation
expressons in `$localize` tagged strings. But there are also potentially
dynamic values in ICU expressions too, so we need to render these as
placeholders when extracting i18n messages into translation files.
PR Close#38484
Previously, the i18n message extractor just quietly ignored messages that
it extracted that had the same id. It can be helpful to identify these
to track down messages that have the same id but different message text.
Now the messages are checked for duplicate ids with different message text.
Any that are found can be reported based on the new `--duplicateMessageHandling`
command line option (or `duplicateMessageHandling` API options property).
* "ignore" - no action is taken
* "warning" - a diagnostic warning is written to the logger
* "error" - the extractor throws an error and exits
Fixes#38077
PR Close#38082
The `fs.relative()` method assumed that the file-system is a single tree,
which is not the case in Windows, where you can have multiple drives,
e.g. `C:`, `D:` etc.
This commit changes `fs.relative()` so that it no longer forces the result
to be a `PathSegment` and then flows that refactoring through the rest of
the compiler-cli (and ngcc). The main difference is that now, in some cases,
we needed to check whether the result is "rooted", i.e an `AbsoluteFsPath`,
rather than a `PathSegment`, before using it.
Fixes#36777
PR Close#37959
When loading a translation file we ask each `TranslationParser`
whether it can parse the file. Occasionally, this check can find
errors in the file that would be useful to the developer. For example
if the file has invalid XML.
This commit deprecates the previous `canParse()` method and replaces it
with a new `analyze()` method. This returns an object that includes a
boolean `canParse` and then either a `hint` if it can parse the file,
or a `diagnostics` object filled with any messages that can be used to
diagnose problems with the format of the file.
Closes #37901
PR Close#37909
This tool, which can be run from the node_modules bin folder, can parse
the source files in your compiled app and generate a translation file
formatted with the configured syntax.
For example:
```
./node_modules/.bin/localize-extract -s 'dist/**/*.js' -f xliff1 -o dist/messages.en.xlf
```
PR Close#32912
Previously source locations required an ending position but this was not
being computed effectively. Now ending position is optional and it is
computed from an `endPath` passed to `getLocation()`.
PR Close#32912
Due to an outage with the proxy we rely on for publishing, we need
to temporarily directly publish to NPM using our own angular
credentials again.
PR Close#37378
This commit makes the leap from its own custom baked `FileUtils`
solution to the fully formed `FileSystem` that is used in the compiler-cli.
This makes testing more straightforward and helps to ensure that the tool
will work across operatings systems.
Also, going forward, it will allow the localize project access to other useful
code from the compiler-cli, such as source-map handling.
PR Close#36843
We can remove all of the entry point resolution configuration from the package.json
in our source code as ng_package rule adds the properties automatically and correctly
configures them.
This change simplifies our code base but doesn't have any impact on the package.json
in the distributed npm_packages.
PR Close#36944
The `getLocation()` method was not working as there were typos in the
properties it was reading. This was not picked up because there were
neither typings for these properties nor unit tests to check it worked.
PR Close#36853
Previously only one translation file per locale could be loaded.
Now the user can specify multiple files per locale, and the translations
from each of these files will be merged together by message id.
The merging is on a first-wins approach. So if to you have three files
to be merged:
```
['a.xlf', 'b.xmb', 'c.json']
```
Then any message from `a.xlf` will be used rather than a message from `b.xmb`
or `c.json` and so on. In practice this means that you should put the files
in order of most important first, with "fallback" translations later.
PR Close#36792
This commit introduces `Diagnostics.merge(other)` which will take the messages from
`other` and append them to the messages of `this`.
The translation loader is updated to use this new method.
PR Close#36792
Previously the `missingTranslation` option had to be checked before calling
`warn` or `error` on the `diagnostics` object.
Now this boilerplate is hidden inside the `Diagnostics.add()` method, which
will open it up to being used for other conditional diagnostics.
PR Close#36792
Previously, we only displayed the new `$localize` id, which is not
currently what most people have in their translation files.
Until people migrate to the new message id system it is confusing
not to display the legacy ids.
PR Close#36761
ts-api-guardian uses `require.resolve` to resolve the actual and golden files under bazel. In Windows for these files to be resolved correct the full path including the workspace name as per the MANIFEST entries is required.
This used to be the case until the recent changes done to use npm_integration tests
83c74ceacf/tools/public_api_guard/public_api_guard.bzl (L19)83c74ceacf/tools/public_api_guard/public_api_guard.bzl (L28)
```
bazel test //packages/... --test_tag_filters=api_guard
//packages/animations:animations_api (cached) PASSED in 18.4s
//packages/common:common_api (cached) PASSED in 25.5s
//packages/compiler-cli:compiler_options_api (cached) PASSED in 12.4s
//packages/compiler-cli:error_code_api (cached) PASSED in 11.6s
//packages/core:core_api (cached) PASSED in 20.6s
//packages/core:ng_global_utils_api (cached) PASSED in 13.5s
//packages/elements:elements_api (cached) PASSED in 11.9s
//packages/forms:forms_api (cached) PASSED in 13.9s
//packages/http:http_api (cached) PASSED in 14.8s
//packages/localize:localize_api (cached) PASSED in 6.3s
//packages/platform-browser:platform-browser_api (cached) PASSED in 18.1s
//packages/platform-browser-dynamic:platform-browser-dynamic_api (cached) PASSED in 14.0s
//packages/platform-server:platform-server_api (cached) PASSED in 13.9s
//packages/platform-webworker:platform-webworker_api (cached) PASSED in 13.7s
//packages/platform-webworker-dynamic:platform-webworker-dynamic_api (cached) PASSED in 11.7s
//packages/router:router_api (cached) PASSED in 19.9s
//packages/service-worker:service-worker_api (cached) PASSED in 18.1s
//packages/upgrade:upgrade_api (cached) PASSED in 13.5s
```
Reference: DEV-71
PR Close#36034
Moves the public api .d.ts files from tools/public_api_guard to
goldens/public-api.
Additionally, provides a README in the goldens directory and a script
assist in testing the current state of the repo against the goldens as
well as a command for accepting all changes to the goldens in a single
command.
PR Close#35768
XLIFF translation files can contain multiple `<file>` elements,
each of which contains translations. In ViewEngine all these
translations are merged into a single translation bundle.
Previously in Ivy only the translations from the last `<file>`
element were being loaded. Now all the translations from each
`<file>` are merged into a single translation bundle.
Fixes#35839
PR Close#35936
This commit improves the `canParse()` method to check that the file is
valid XML and has the expected root node. Previously it was relying upon
a regular expression to do this.
PR Close#35793
Previously, the `Xliff2TranslationParser` only matched files that had a narrow
choice of extensions (e.g. `xlf`) and also relied upon a regular expression
match of an optional XML namespace directive.
This commit relaxes the requirement on both of these and, instead, relies
upon parsing the file into XML and identifying an element of the form
`<xliff version="2.0">` which is the minimal requirement for such files.
PR Close#35793
Previously, the `Xliff1TranslationParser` only matched files that had a narrow
choice of extensions (e.g. `xlf`) and also relied upon a regular expression
match of an optional XML namespace directive.
This commit relaxes the requirement on both of these and, instead, relies
upon parsing the file into XML and identifying an element of the form
`<xliff version="1.2">` which is the minimal requirement for such files.
PR Close#35793
Pete Bacon Darwin
Joey Perrott
Igor Minar
Walter Werner SCHNEIDER
Alan Agius
Joey Perrott