Commit Graph

116 Commits

Author SHA1 Message Date
Alex Rickabaugh 3613e7c4e5 test(compiler-cli): move testing utils to separate package (#39594)
ngtsc has a robust suite of testing utilities, designed for in-memory
testing of a TypeScript compiler. Previously these utilities lived in the
`test` directory for the compiler-cli package.

This commit moves those utilities to an `ngtsc/testing` package, enabling
them to be depended on separately and opening the door for using them from
the upcoming language server testing infrastructure.

As part of this refactoring, the `fake_core` package (a lightweight API
replacement for @angular/core) is expanded to include functionality needed
for Language Service test use cases.

PR Close #39594
2020-11-17 11:59:56 -08:00
Pete Bacon Darwin 0ed2c4fef9 fix(localize): render placeholder types in extracted XLIFF files (#39398)
The previous ViewEngine extraction tooling added `ctype` and `type`
attributes to XLIFF 1.2 and 2.0 translation files, respectively.

This commit adds this to the new $localize based extraction tooling.
Since the new extraction tooling works from the compiled output rather
than having direct access to the template content, the placeholder types
must be inferred from the name of the placeholder.  This is considered
reasonable, since it already does this to compute opening and closing
tag placeholders.

Fixes #38791

PR Close #39398
2020-10-27 10:42:15 -07:00
Pete Bacon Darwin 57ead7aa53 fix(localize): serialize all the message locations to XLIFF (#39411)
Previously only the first message, for each id, was serialized
which meant that additional message location information
was lost.

Now all the message locations are included in the serialized
messages.

Fixes #39330

PR Close #39411
2020-10-26 10:52:29 -07:00
Greg Magolan 42a164f522 build: upgrade angular build, integration/bazel and @angular/bazel package to rule_nodejs 2.2.0 (#39182)
Updates to rules_nodejs 2.2.0. This is the first major release in 7 months and includes a number of features as well
as breaking changes.

Release notes: https://github.com/bazelbuild/rules_nodejs/releases/tag/2.0.0

Features of note for angular/angular:

* stdout/stderr/exit code capture; this could be potentially be useful

* TypeScript (ts_project); a simpler tsc rule that ts_library that can be used in the repo where ts_library is too
  heavy weight

Breaking changes of note for angular/angular:

* loading custom rules from npm packages: `ts_library` is no longer loaded from `@npm_bazel_typescript//:index.bzl`
  (which no longer exists) but is now loaded from `@npm//@bazel/typescript:index.bzl`

* with the loading changes above, `load("@npm//:install_bazel_dependencies.bzl", "install_bazel_dependencies")` is
  no longer needed in the WORKSPACE which also means that yarn_install does not need to run unless building/testing
  a target that depends on @npm. In angular/angular this is a minor improvement as almost everything depends on @npm.

* @angular/bazel package is also updated in this PR to support the new load location; Angular + Bazel users that
  require it for ng_package (ng_module is no longer needed in OSS with Angular 10) will need to load from
  `@npm//@angular/bazel:index.bzl`. I investigated if it was possible to maintain backward compatability for the old
  load location `@npm_angular_bazel` but it is not since the package itself needs to be updated to load from
  `@npm//@bazel/typescript:index.bzl` instead of `@npm_bazel_typescript//:index.bzl` as it depends on ts_library
  internals for ng_module.

* runfiles.resolve will now throw instead of returning undefined to match behavior of node require

Other changes in angular/angular:

* integration/bazel has been updated to use both ng_module and ts_libary with use_angular_plugin=true.
  The latter is the recommended way for rules_nodejs users to compile Angular 10 with Ivy. Bazel + Angular ViewEngine is
  supported with @angular/bazel <= 9.0.5 and Angular <= 8. There is still Angular ViewEngine example on rules_nodejs
  https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular_view_engine on these older versions but users
  that want to update to Angular 10 and are on Bazel must switch to Ivy and at that point ts_library with
  use_angular_plugin=true is more performant that ng_module. Angular example in rules_nodejs is configured this way
  as well: https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular. As an aside, we also have an
  example of building Angular 10 with architect() rule directly instead of using ts_library with angular plugin:
  https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular_bazel_architect.

NB: ng_module is still required for angular/angular repository as it still builds ViewEngine & @angular/bazel
also provides the ng_package rule. ng_module can be removed in the future if ViewEngine is no longer needed in
angular repo.

* JSModuleInfo provider added to ng_module. this is for forward compat for future rules_nodejs versions.

PR Close #39182
2020-10-08 11:54:59 -07:00
Joey Perrott c214cad2b4 Revert "build: upgrade angular build, integration/bazel and @angular/bazel package to rule_nodejs 2.2.0 (#37727)" (#39097)
This reverts commit db56cf18ba.

PR Close #39097
2020-10-02 10:56:53 -07:00
Greg Magolan db56cf18ba build: upgrade angular build, integration/bazel and @angular/bazel package to rule_nodejs 2.2.0 (#37727)
Updates to rules_nodejs 2.2.0. This is the first major release in 7 months and includes a number of features as well
as breaking changes.

Release notes: https://github.com/bazelbuild/rules_nodejs/releases/tag/2.0.0

Features of note for angular/angular:

* stdout/stderr/exit code capture; this could be potentially be useful

* TypeScript (ts_project); a simpler tsc rule that ts_library that can be used in the repo where ts_library is too
  heavy weight

Breaking changes of note for angular/angular:

* loading custom rules from npm packages: `ts_library` is no longer loaded from `@npm_bazel_typescript//:index.bzl`
  (which no longer exists) but is now loaded from `@npm//@bazel/typescript:index.bzl`

* with the loading changes above, `load("@npm//:install_bazel_dependencies.bzl", "install_bazel_dependencies")` is
  no longer needed in the WORKSPACE which also means that yarn_install does not need to run unless building/testing
  a target that depends on @npm. In angular/angular this is a minor improvement as almost everything depends on @npm.

* @angular/bazel package is also updated in this PR to support the new load location; Angular + Bazel users that
  require it for ng_package (ng_module is no longer needed in OSS with Angular 10) will need to load from
  `@npm//@angular/bazel:index.bzl`. I investigated if it was possible to maintain backward compatability for the old
  load location `@npm_angular_bazel` but it is not since the package itself needs to be updated to load from
  `@npm//@bazel/typescript:index.bzl` instead of `@npm_bazel_typescript//:index.bzl` as it depends on ts_library
  internals for ng_module.

* runfiles.resolve will now throw instead of returning undefined to match behavior of node require

Other changes in angular/angular:

* integration/bazel has been updated to use both ng_module and ts_libary with use_angular_plugin=true.
  The latter is the recommended way for rules_nodejs users to compile Angular 10 with Ivy. Bazel + Angular ViewEngine is
  supported with @angular/bazel <= 9.0.5 and Angular <= 8. There is still Angular ViewEngine example on rules_nodejs
  https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular_view_engine on these older versions but users
  that want to update to Angular 10 and are on Bazel must switch to Ivy and at that point ts_library with
  use_angular_plugin=true is more performant that ng_module. Angular example in rules_nodejs is configured this way
  as well: https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular. As an aside, we also have an
  example of building Angular 10 with architect() rule directly instead of using ts_library with angular plugin:
  https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular_bazel_architect.

NB: ng_module is still required for angular/angular repository as it still builds ViewEngine & @angular/bazel
also provides the ng_package rule. ng_module can be removed in the future if ViewEngine is no longer needed in
angular repo.

* JSModuleInfo provider added to ng_module. this is for forward compat for future rules_nodejs versions.
  @josephperrott, this touches `packages/bazel/src/external.bzl` which will make the sync to g3 non-trivial.

PR Close #37727
2020-10-01 15:34:36 -07:00
Pete Bacon Darwin 79620f5139 refactor(localize): avoid free-standing `FileSystem` functions (#39006)
These free standing functions rely upon the "current" `FileSystem`,
but it is safer to explicitly pass the `FileSystem` into functions or
classes that need it.

Fixes #38711

PR Close #39006
2020-09-30 12:49:43 -07:00
Pete Bacon Darwin 23e2188192 docs(localize): add public api markers for CLI integration (#39006)
This commit marks the functions and classes that are
used by the CLI.

PR Close #39006
2020-09-30 12:49:43 -07:00
Pete Bacon Darwin ce1efc1af2 fix(localize): ensure that `formatOptions` is optional (#38787)
Some lower level APIs are used by CLI, and requiring
the `formatOpions` argument at that level is a
breaking change. This commit makes it optional
at every level to avoid the breaking change.

PR Close #38787
2020-09-10 10:55:04 -07:00
Pete Bacon Darwin 4360eed9b7 fix(localize): enable whitespace preservation marker in XLIFF files (#38737)
Whitespace can be relevant in extracted XLIFF translation files.
Some i18n tools - e.g. CAT tool (OmegaT) - will reformat
the file to collapse whitespace if there is no indication to tell it
not to.

This commit adds the ability to specify "format options" that are passed
to the translation file serializer. The XLIFF 1.2 and 2.0 seralizers have
been updated to accept `{"xml:space":"preserve"}` format option which will
by added to the `<file>` element in the serialized translation file during
extraction.

Fixes #38679

PR Close #38737
2020-09-08 14:24:51 -07:00
Pete Bacon Darwin 2c4a98a285 fix(localize): do not expose NodeJS typings in $localize runtime code (#38700)
A recent change to `@angular/localize` brought in the `AbsoluteFsPath` type
from the `@angular/compiler-cli`. But this brought along with it a reference
to NodeJS typings - specifically the `FileSystem` interface refers to the
`Buffer` type from NodeJS.

This affects compilation of `@angular/localize` code that will be run in
the browser - for example projects that reference `loadTranslations()`.
The compilation breaks if the NodeJS typings are not included in the build.
Clearly it is not desirable to have these typings included when the project
is not targeting NodeJS.

This commit replaces references to the NodeJS `Buffer` type with `Uint8Array`,
which is available across all platforms and is actually the super-class of
`Buffer`.

Fixes #38692

PR Close #38700
2020-09-08 11:40:58 -07:00
Pete Bacon Darwin 92ff6d93eb fix(localize): render location in XLIFF 2 even if there is no metadata (#38713)
Previously, the location of a translation message, in XLIFF 2, was only
rendered if there were also notes for meaning or description. Now the
location will be rendered even if the other metadata is not provided.

Fixes #38705

PR Close #38713
2020-09-08 11:40:35 -07:00
Pete Bacon Darwin 50f4d8a1ce fix(localize): install `@angular/localize` in `devDependencies` by default (#38680)
Previously this package was installed in the default `dependencies` section
of `package.json`, but this meant that its own dependencies are treated as
dependencies of the main project: Babel, for example.

Generally, $localize` is not used at runtime - it is compiled out by the
translation tooling, so there is no need for it to be a full dependency.
In fact, even if it is used at runtime, the package itself is only used
at dev-time since the runtime bits will be bundled into a distributable.
So putting this package in `devDependencies` would only prevent libraries
from bringing the package into application projects that used them. This
is probably good in itself, since it should be up to the downstream project
to decide if it wants to include `@angular/localize` at runtime.

This commit changes the default location of the package to be the
`devDependencies` section, but gives an option `useAtRuntime` to choose
otherwise.

Fixes #38329

PR Close #38680
2020-09-03 09:41:38 -07:00
Pete Bacon Darwin a68f1a78a7 refactor(compiler): element.startSourceSpan is required (#38581)
Previously, the `startSourceSpan` property could be null
but in reality it is always well defined - except for a legacy
case in the old i18n extraction/merging code, where the
typings for source-spans are already being undermined.

Making this property non-null, simplifies code elsewhere
in the project.

PR Close #38581
2020-09-02 14:47:28 -07:00
Pete Bacon Darwin 5da1934115 fix(localize): render context of translation file parse errors (#38673)
Previously the position of the error in a translation file when parsing
it was not displayed. Just the error message.

Now the position (line and column) and some context is displayed
along with the error messages.

Fixes #38377

PR Close #38673
2020-09-02 14:46:13 -07:00
Pete Bacon Darwin f0af387f6c fix(localize): ensure required XLIFF parameters are serialized (#38575)
When extracting i18n messages from source code, the XLIFF
serializers were missing some required attributes on the `<file>`
element.

This commit re-introduces the `original` property to each of XLIFF 1.2
and 2.0 serializers. Also it adds in the required `id` property for the
XLIFF 2.0 seralizer.

Fixes #38570

PR Close #38575
2020-08-25 15:14:19 -07:00
Pete Bacon Darwin 14e90bef58 fix(localize): render text of extracted placeholders (#38536)
Formats like XLIFF allow the text of the original source to
be included as metadata. This commit fixes the message
extractor to also render this text when available.

PR Close #38536
2020-08-25 15:13:23 -07:00
Pete Bacon Darwin db3a21b382 refactor(localize): add placeholder locations in extracted messages (#38536)
Some translation file formats would like to be able to render the
text of placeholders taken from the original source files. This commit
adds this information to the extracted messages so that it can be
used in translation file serializers.

PR Close #38536
2020-08-25 15:13:20 -07:00
Pete Bacon Darwin b8351f3b10 refactor(localize): ensure that translate plugin exceptions are not swallowed (#38536)
Previously, exceptions that were not `BabelParseError`s were just ignored.
Such exceptions are most likely programming errors in the package.
They are now re-thrown to ensure that the error is not hidden.

PR Close #38536
2020-08-25 15:13:17 -07:00
Pete Bacon Darwin 81053d3160 refactor(localize): run the translate plugin tests in mock FileSystems (#38536)
This commit is a tidy up of the translate plugin unit tests, but also ensures
that the tests are run in the context of a mock FileSystem. This ensures
that the tests are resilient to future refactors of the plugins that will
require a FileSystem to be initialized.

PR Close #38536
2020-08-25 15:13:15 -07:00
Pete Bacon Darwin bdba1a062d refactor(localize): include text in original location of extracted messages (#38536)
When extracting messages, source-mapping information is used to find
the original location of the message being extracted. This commit will
now include the text from the original source in the message location
so that it can be serialized into the translation file.

PR Close #38536
2020-08-25 15:13:12 -07:00
Pete Bacon Darwin 23f855b300 refactor(localize): allow ParsedMessage to hold additional location data (#38536)
In preparation for supporting `equiv-text` placeholder information in
extracted translation files, this commit adds these optional properties
to the `ParsedMessage` interface and updates `parseMessage()` to
be able to store them.

PR Close #38536
2020-08-25 15:13:09 -07:00
Pete Bacon Darwin ac461e1efd fix(localize): extract the correct message ids (#38498)
Previously, if `useLegacyIds` was enabled, the message extractor
was always rendering the legacy message ids in translation
files even if an explicit "custom message id" had been provided
in the original message.

PR Close #38498
2020-08-19 14:19:41 -07:00
Pete Bacon Darwin 68a9a01a64 fix(localize): parse all parts of a translation with nested HTML (#38452)
Previously nested container placeholders (i.e. HTML elements) were
not being fully parsed from translation files. This resulted in bad
translation of messages that contain these placeholders.

Note that this causes the canonical message ID to change for
such messages. Currently all messages generated from
templates use "legacy" message ids that are not affected by
this change, so this fix should not be seen as a breaking change.

Fixes #38422

PR Close #38452
2020-08-19 14:16:41 -07:00
Pete Bacon Darwin 8cd4099db9 fix(localize): include the last placeholder in parsed translation text (#38452)
When creating a `ParsedTranslation` from a set of message parts and
placeholder names a textual representation of the message is computed.
Previously the last placeholder and text segment were missing from this
computed message string.

PR Close #38452
2020-08-19 14:16:38 -07:00
Joey Perrott 8373b720f3 refactor(localize): update yargs and typings for yargs (#38470)
Updating yargs and typings for the updated yargs module.

PR Close #38470
2020-08-17 15:30:32 -07:00
Pete Bacon Darwin 81c3e809aa fix(localize): render ICU placeholders in extracted translation files (#38484)
Previously placeholders were only rendered for dynamic interpolation
expressons in `$localize` tagged strings. But there are also potentially
dynamic values in ICU expressions too, so we need to render these as
placeholders when extracting i18n messages into translation files.

PR Close #38484
2020-08-17 10:44:26 -07:00
Pete Bacon Darwin cf9a47ba53 feat(localize): allow duplicate messages to be handled during extraction (#38082)
Previously, the i18n message extractor just quietly ignored messages that
it extracted that had the same id. It can be helpful to identify these
to track down messages that have the same id but different message text.

Now the messages are checked for duplicate ids with different message text.
Any that are found can be reported based on the new `--duplicateMessageHandling`
command line option (or `duplicateMessageHandling` API options property).

* "ignore" - no action is taken
* "warning" - a diagnostic warning is written to the logger
* "error" - the extractor throws an error and exits

Fixes #38077

PR Close #38082
2020-07-16 13:53:38 -07:00
Pete Bacon Darwin 56dd3e77ac refactor(localize): remove unnecessary ! operator (#38082)
The messages property always exists so there is no need for the ! operator.

PR Close #38082
2020-07-16 13:53:38 -07:00
Pete Bacon Darwin 6b311552f0 fix(compiler-cli): ensure file_system handles mixed Windows drives (#37959)
The `fs.relative()` method assumed that the file-system is a single tree,
which is not the case in Windows, where you can have multiple drives,
e.g. `C:`, `D:` etc.

This commit changes `fs.relative()` so that it no longer forces the result
to be a `PathSegment` and then flows that refactoring through the rest of
the compiler-cli (and ngcc).  The main difference is that now, in some cases,
we needed to check whether the result is "rooted", i.e an `AbsoluteFsPath`,
rather than a `PathSegment`, before using it.

Fixes #36777

PR Close #37959
2020-07-13 12:05:21 -07:00
Pete Bacon Darwin ec32eba02c feat(localize): expose `canParse()` diagnostics (#37909)
When loading a translation file we ask each `TranslationParser`
whether it can parse the file. Occasionally, this check can find
errors in the file that would be useful to the developer. For example
if the file has invalid XML.

This commit deprecates the previous `canParse()` method and replaces it
with a new `analyze()` method. This returns an object that includes a
boolean `canParse` and then either a `hint` if it can parse the file,
or a `diagnostics` object filled with any messages that can be used to
diagnose problems with the format of the file.

Closes #37901

PR Close #37909
2020-07-06 13:56:39 -07:00
Pete Bacon Darwin 190561d8a6 feat(localize): implement message extraction tool (#32912)
This tool, which can be run from the node_modules bin folder, can parse
the source files in your compiled app and generate a translation file
formatted with the configured syntax.

For example:

```
./node_modules/.bin/localize-extract -s 'dist/**/*.js' -f xliff1 -o dist/messages.en.xlf
```

PR Close #32912
2020-06-25 14:10:03 -07:00
Pete Bacon Darwin ddb0a4e2e5 refactor(localize): handle location endings correctly (#32912)
Previously source locations required an ending position but this was not
being computed effectively. Now ending position is optional and it is
computed from an `endPath` passed to `getLocation()`.

PR Close #32912
2020-06-25 14:10:03 -07:00
Joey Perrott 0a1d078a74 Revert "build: remove wombot proxy registry from package.jsons for release (#37378)" (#37495)
This reverts commit 26849ca99d.

PR Close #37495
2020-06-10 08:21:45 -07:00
Joey Perrott 26849ca99d build: remove wombot proxy registry from package.jsons for release (#37378)
Due to an outage with the proxy we rely on for publishing, we need
to temporarily directly publish to NPM using our own angular
credentials again.

PR Close #37378
2020-06-01 12:41:19 -04:00
Joey Perrott d1ea1f4c7f build: update license headers to reference Google LLC (#37205)
Update the license headers throughout the repository to reference Google LLC
rather than Google Inc, for the required license headers.

PR Close #37205
2020-05-26 14:26:58 -04:00
Pete Bacon Darwin 141fcb95a4 refactor(localize): use the `FileSystem` from ngtsc (#36843)
This commit makes the leap from its own custom baked `FileUtils`
solution to the fully formed `FileSystem` that is used in the compiler-cli.

This makes testing more straightforward and helps to ensure that the tool
will work across operatings systems.

Also, going forward, it will allow the localize project access to other useful
code from the compiler-cli, such as source-map handling.

PR Close #36843
2020-05-13 15:52:48 -07:00
Igor Minar d578ab8f3c build: simplify package.jsons for all of our packages (#36944)
We can remove all of the entry point resolution configuration from the package.json
in our source code as ng_package rule adds the properties automatically and correctly
configures them.

This change simplifies our code base but doesn't have any impact on the package.json
in the distributed npm_packages.

PR Close #36944
2020-05-06 13:54:26 -07:00
Walter Werner SCHNEIDER 1c26f40cd4 refactor(localize): use the new workspaces API for ng-add schematic (#36897)
Updates the @angular/localize ng-add schematic to use the new workspaces API and removes dependency on private APIs.

PR Close #36897
2020-05-05 11:58:27 -07:00
Pete Bacon Darwin 70b25a3d4f fix(localize): ensure `getLocation()` works (#36853)
The `getLocation()` method was not working as there were typos in the
properties it was reading. This was not picked up because there were
neither typings for these properties nor unit tests to check it worked.

PR Close #36853
2020-05-05 11:51:46 -07:00
Pete Bacon Darwin 72f534f7f8 feat(localize): support merging multiple translation files (#36792)
Previously only one translation file per locale could be loaded.

Now the user can specify multiple files per locale, and the translations
from each of these files will be merged together by message id.
The merging is on a first-wins approach. So if to you have three files
to be merged:

```
['a.xlf', 'b.xmb', 'c.json']
```

Then any message from `a.xlf` will be used rather than a message from `b.xmb`
or `c.json` and so on. In practice this means that you should put the files
in order of most important first, with "fallback" translations later.

PR Close #36792
2020-05-01 09:46:12 -07:00
Pete Bacon Darwin 3cb9b43851 refactor(localize): hide merging diagnostics messages behind a method (#36792)
This commit introduces `Diagnostics.merge(other)` which will take the messages from
`other` and append them to the messages of `this`.

The translation loader is updated to use this new method.

PR Close #36792
2020-05-01 09:46:12 -07:00
Pete Bacon Darwin bf0c520f2e refactor(localize): simplify adding condition diagnostics (#36792)
Previously the `missingTranslation` option had to be checked before calling
`warn` or `error` on the `diagnostics` object.

Now this boilerplate is hidden inside the `Diagnostics.add()` method, which
will open it up to being used for other conditional diagnostics.

PR Close #36792
2020-05-01 09:46:11 -07:00
Pete Bacon Darwin 113c75dd27 refactor(localize): remove unused code (#36745)
PR Close #36745
2020-04-23 12:07:33 -04:00
Pete Bacon Darwin b7acf07a70 refactor(localize): consolidate message/translation metadata (#36745)
PR Close #36745
2020-04-23 12:07:33 -04:00
Pete Bacon Darwin 519f2baff0 test(localize): tidy up translation parser tests (#36745)
There was a lot of duplication and multiline backtick
strings that made it hard to maintain.

PR Close #36745
2020-04-23 12:07:33 -04:00
Pete Bacon Darwin 3962908d36 refactor(localize): tighten up recognition of simple JSON translations (#36745)
PR Close #36745
2020-04-23 12:07:33 -04:00
Pete Bacon Darwin afd2cbc3c4 refactor(localize): move `source_file_utils.ts` up (#36745)
This will allow the utilities in this file to be shared outside
`translate` code.

PR Close #36745
2020-04-23 12:07:33 -04:00
Pete Bacon Darwin 47f98673c7 fix(localize): include legacy ids when describing messages (#36761)
Previously, we only displayed the new `$localize` id, which is not
currently what most people have in their translation files.
Until people migrate to the new message id system it is confusing
not to display the legacy ids.

PR Close #36761
2020-04-22 16:31:47 -04:00
Joey Perrott 698b0288be build: reformat repo to new clang@1.4.0 (#36613)
PR Close #36613
2020-04-14 12:08:36 -07:00