Commit Graph

60 Commits

Author SHA1 Message Date
Naarcha-AWS 72b3363460
Add codec and processor combinations to S3 source (#5288)
* Add codec and processor combinations to S3 source.

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Update s3.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Make codec-processor combinations its own file

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Make Data Prepper codec page

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Fix links

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Fix links

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Reformat article without tables.

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* fix link

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/common-use-cases/codec-processor-combinations.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update codec-processor-combinations.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
2023-11-17 20:06:04 -06:00
kolchfa-aws 3c94bc80d7
Fix links and liquid formatting error (#5512)
Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>
2023-11-06 12:23:59 -05:00
Qi Chen 96175482d6
[DOC] data prepper secret extensions (#5202)
* ADD: extension docs in data-prepper-config

Signed-off-by: George Chen <qchea@amazon.com>

* MAINT: updating secrets extension doc

Signed-off-by: George Chen <qchea@amazon.com>

* MAINT: fix links

Signed-off-by: George Chen <qchea@amazon.com>

* MAINT: fix one more dead link

Signed-off-by: George Chen <qchea@amazon.com>

* MAINT: renaming

Signed-off-by: George Chen <qchea@amazon.com>

* Update configuring-data-prepper.md

* Update pipelines.md

---------

Signed-off-by: George Chen <qchea@amazon.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
2023-11-03 12:08:00 -05:00
Samuel Valdes Gutierrez 7012af124a
[OSCI][DOCs] Replacing 'indices' terms for 'indexes' terms ONLY for description texts (#5353)
* Fixing documentation for Wildcard in term-level queries section for Query DSL

Signed-off-by: Samuel Valdes Gutierrez <valdesgutierrez@gmail.com>

* replacing 'indices' term for 'indexes' term ONLY for description texts (not variables, links or properties)

Signed-off-by: Samuel Valdes Gutierrez <valdesgutierrez@gmail.com>

* Update creating-custom-workloads.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* updating changes suggested by Naarcha-AWS

Signed-off-by: Samuel Valdes Gutierrez <valdesgutierrez@gmail.com>

* updating changes suggested by Naarcha-AWS

Signed-off-by: Samuel Valdes Gutierrez <valdesgutierrez@gmail.com>

* Rename _benchmark/workloads/index.md to _benchmark/workloads/reference/index.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Rename _benchmark/workloads/indices.md to _benchmark/workloads/reference/indices.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Samuel Valdes Gutierrez <valdesgutierrez@gmail.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
2023-10-31 10:18:45 -05:00
Hai Yan 03d177aa2b
Add documentation for Data Prepper translate processor (#5122)
* Add translate processor docs

Signed-off-by: Hai Yan <oeyh@amazon.com>

* Fix a table display issue

Signed-off-by: Hai Yan <oeyh@amazon.com>

* Address review comments: tweak sentences; add type info; add top level options

Signed-off-by: Hai Yan <oeyh@amazon.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/translate.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update translate.md

* Update translate.md

* Update _data-prepper/pipelines/configuration/processors/translate.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Hai Yan <oeyh@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-10-12 08:53:20 -05:00
Taylor Gray f4984fb605
Documentation for the Data Prepper OpenSearch source (#5144)
* Documentation for the Data Prepper OpenSearch source

Signed-off-by: Taylor Gray <tylgry@amazon.com>

* Update _data-prepper/pipelines/configuration/sources/opensearch.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Taylor Gray <graytaylor0@gmail.com>

* Update opensearch.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/opensearch.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/opensearch.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update opensearch.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/opensearch.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Taylor Gray <graytaylor0@gmail.com>

* Update _data-prepper/pipelines/configuration/sources/opensearch.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Taylor Gray <graytaylor0@gmail.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Taylor Gray <tylgry@amazon.com>
Signed-off-by: Taylor Gray <graytaylor0@gmail.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-10-11 13:48:53 -05:00
Hai Yan 55fb97b290
Update Data Prepper key_value processor docs (#5153)
* Add new options to table

Signed-off-by: Hai Yan <oeyh@amazon.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Hai Yan <oeyh@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-10-11 10:00:57 -05:00
Hai Yan 66db871345
Add docs for Data Prepper dissect processor (#5159)
* Add docs for dissect processor

Signed-off-by: Hai Yan <oeyh@amazon.com>

* Fix some style issues

Signed-off-by: Hai Yan <oeyh@amazon.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/dissect.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Hai Yan <oeyh@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-10-11 10:00:28 -05:00
Jonah Calvo 7eceb2baf0
Update documentation for new AD settings (#4835)
* Update documentation for new AD settings

Signed-off-by: Jonah Calvo <caljonah@amazon.com>

* update wording for verbose

Signed-off-by: Jonah Calvo <caljonah@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
Signed-off-by: Jonah Calvo <jonah.calvo@gmail.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
Signed-off-by: Jonah Calvo <jonah.calvo@gmail.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
Signed-off-by: Jonah Calvo <jonah.calvo@gmail.com>

* Remove 'few' from description

Signed-off-by: Jonah Calvo <caljonah@amazon.com>

---------

Signed-off-by: Jonah Calvo <caljonah@amazon.com>
Signed-off-by: Jonah Calvo <jonah.calvo@gmail.com>
Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
2023-09-21 15:07:38 -05:00
Vlad Babak 0e2a5888a9
Typo in convert_entry_type.md (#4994)
Replaced "convert_entry_type_type" with "convert_entry_type".

Signed-off-by: Vlad Babak <129367331+vlad-voxalyze@users.noreply.github.com>
2023-09-08 14:03:14 -05:00
Naarcha-AWS ea886a4763
Fix S3 bug (#4974)
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
2023-09-06 16:01:03 -05:00
David Venable 64d59b9bb2
Behavioral changes for Data Prepper S3 sink (#4897)
* Updates the Data Prepper documentation for S3 sinks based on recent behavior changes.

Signed-off-by: David Venable <dlv@amazon.com>

* Updates from the PR feedback.

Signed-off-by: David Venable <dlv@amazon.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: David Venable <dlv@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
2023-08-29 10:01:24 -05:00
David Venable 27cbf9d03a
Removes the acknowledgements_timeout configuration from the Data Prepper Kafka source. (#4898)
Signed-off-by: David Venable <dlv@amazon.com>
2023-08-28 11:47:36 -05:00
David Venable 000aecc1a3
Moves the generic sink options into the Sinks documentation rather than PIpeline Configuration options. (#4907)
Signed-off-by: David Venable <dlv@amazon.com>
2023-08-28 10:43:26 -05:00
David Venable 5306134c41
Updated the intro text for the s3 sink. (#4854)
Signed-off-by: David Venable <dlv@amazon.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
2023-08-23 13:33:46 -05:00
Naarcha-AWS 1250649984
Fix typos on S3 sink documentation (#4862)
* Fix typos on S3 sink documentation

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sinks/s3.md

Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>
2023-08-23 12:49:46 -05:00
David Venable ed21ec74cf
Removes the Data Prepper csv sink codec. This is not ready for use and should not be documented. (#4859)
Signed-off-by: David Venable <dlv@amazon.com>
2023-08-23 09:19:04 -05:00
Asif Sohail Mohammed cc21df7ee8
S3 scan doc update (#4848)
* Updated s3 scan docs

Signed-off-by: Asif Sohail Mohammed <nsifmoh@amazon.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Asif Sohail Mohammed <nsifmoh@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
2023-08-22 15:28:13 -05:00
David Venable 0b5011409a
Data Prepper documentation updates for the s3 sink in 2.4.0 (#4847)
* Data Prepper documentation updates for the s3 sink in 2.4.0. Includes other corrections.

* Updated the Parquet/Avro codecs to include auto_schema as well as some clarifications on the restrictions of the auto-generated schema.

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
2023-08-22 15:18:40 -05:00
Asif Sohail Mohammed 8e9d0ed4de
S3 scan docs (#4756)
* Add notification_source option for S3

Signed-off-by: Asif Sohail Mohammed <nsifmoh@amazon.com>

* Added S3 scan configuration

Signed-off-by: Asif Sohail Mohammed <nsifmoh@amazon.com>

* Addressed some failing style checks

Signed-off-by: Asif Sohail Mohammed <nsifmoh@amazon.com>

* Edit S3 additions.

* Fix links.

* Fix style flaw

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update s3.md

---------

Signed-off-by: Asif Sohail Mohammed <nsifmoh@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-08-21 11:02:07 -05:00
David Venable ac3f60d5ca
Documentation for bucket ownership features in Data Prepper 2.4 (#4679)
* Adds documentation for new features in Data Prepper 2.4 related to bucket ownership. Includes a section explaining how the bucket ownership works and can be configured for cross-account access. Resolves #4678

Signed-off-by: David Venable <dlv@amazon.com>

* Updates from PR feedback.

Signed-off-by: David Venable <dlv@amazon.com>

* Remove redundant header

Add a name to primary H2 header to section to prevent links from breaking.

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: David Venable <dlv@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-08-21 10:46:04 -05:00
Qi Chen 7d077786bc
MAINT: update distribution_version in opensearch sink (#4742)
* MAINT: update distribution_version in opensearch sink

Signed-off-by: George Chen <qchea@amazon.com>

* Update _data-prepper/pipelines/configuration/sinks/opensearch.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Qi Chen <chenqi0805@gmail.com>

* Update _data-prepper/pipelines/configuration/sinks/opensearch.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Qi Chen <chenqi0805@gmail.com>

* Fix typo

* ADD: request compression flag

Signed-off-by: George Chen <qchea@amazon.com>

* Update opensearch.md

---------

Signed-off-by: George Chen <qchea@amazon.com>
Signed-off-by: Qi Chen <chenqi0805@gmail.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
2023-08-21 10:45:40 -05:00
Naarcha-AWS 152ffcd79f
Add Kafka source for DP 2.4 (#4737)
* Add Kafka source for DP 2.4

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Use new Kafka settings

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Add technical feedback

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Fix typos

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Apply suggestions from code review

Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM>
Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Change array to object.

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Last minute changes to Kafka defaults

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-08-21 10:43:32 -05:00
David Venable 442f4c446a
Documents general sink options (#4759)
* Adds general sink options which includes: routes, tags_target_key, include_keys, exclude_keys.

Signed-off-by: David Venable <dlv@amazon.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: David Venable <dlv@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
2023-08-21 10:39:31 -05:00
Naarcha-AWS c14b8b2a07
Add CIDR and User Agent processors (#4585)
* Add CIDR and User Agent processors

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Add CIDR functions

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Fix typos

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Add technical feedback

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Update user-agent.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/expression-syntax.md

Co-authored-by: Hai Yan <8153134+oeyh@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update user-agent.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>
Co-authored-by: Hai Yan <8153134+oeyh@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-07-27 14:58:47 -05:00
Naarcha-AWS 0fcfc4862e
Add S3 sink documentation (#4340)
* Add S3 sink documentation

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Update _data-prepper/pipelines/configuration/sinks/s3.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sinks/s3.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sinks/s3.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-07-18 16:06:06 -07:00
Naarcha-AWS adae164d53
Add obfuscate processor (#4515)
* Add obfuscate processor reference

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* fix typo

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Fix paragraph description.

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Fix consistency issues

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Apply suggestions from code review

Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-07-14 12:44:01 -05:00
Powershellfish d78bd5b70e
Update capitalization for Events (#4213)
Signed-off-by: Powershellfish <47839113+powershellfish@users.noreply.github.com>
2023-07-05 13:50:27 -05:00
David Venable 5a4daa4225
Documents the Data Prepper opensearch sink's template_type parameter. (#4290)
* Documents the Data Prepper opensearch sink's template_type parameter.

Signed-off-by: David Venable <dlv@amazon.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: David Venable <dlv@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
2023-06-13 13:53:29 -05:00
Powershellfish 3b9ae603de
Updated capitalization. (#4214)
Signed-off-by: Powershellfish <47839113+powershellfish@users.noreply.github.com>
2023-06-07 06:41:42 -07:00
Powershellfish baa2fb45ef
[DOC] Remove indentation in copy_values processor (#4057)
* Test.

Signed-off-by: Powershellfish <47839113+powershellfish@users.noreply.github.com>

* Update index.md

Testing ability to create PRs.

* Removed indentation.

Signed-off-by: Powershellfish <47839113+powershellfish@users.noreply.github.com>

---------

Signed-off-by: Powershellfish <47839113+powershellfish@users.noreply.github.com>
2023-05-15 17:49:13 -07:00
kolchfa-aws 8463c8f278
Correct plugin capitalization (#3838)
* Correct plugin capitalization

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Revert cluster-stats because the name is in response

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Revert cluster-stats once more

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

---------

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>
2023-05-04 11:11:54 -04:00
Heather Halter f35e72d577
fix list-to-map order (#3941)
Signed-off-by: Heather Halter <hdhalter@amazon.com>
2023-05-03 10:37:27 -07:00
Heather Halter 0235af562f
fix table headings (#3943)
Signed-off-by: Heather Halter <hdhalter@amazon.com>
2023-05-03 07:04:30 -07:00
Naarcha-AWS 1259454b73
Fix typo on processor page. (#3903)
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
2023-04-27 18:58:54 -05:00
Naarcha-AWS 4251fb04bd
Add DLQ and aggregate processor edits. Baseline processor names (#3880)
* Add DLQ and aggregate processor edits. Baseline processor names

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Remove processor from titles

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Update dlq.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Add additional feedback

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* add cross links

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Fix typos

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Add doc review

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Remove link

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Apply suggestions from code review

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update dlq.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Fix processor title

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-27 14:01:01 -05:00
Caroline 66f30d757f
Add DLQ to OpenSearch sink configuration options. (#3743)
* Add DLQ to OpenSearch sink configuration options.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/sinks/opensearch.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
2023-04-26 11:36:47 -05:00
Caroline ab10e6a296
Add OTel trace group processor to doc website. (#3236)
* Add otel trace group processor.

Signed-off-by: carolxob <carolxob@amazon.com>

* Copy edits to improve table formatting.

Signed-off-by: carolxob <carolxob@amazon.com>

* Incorporated doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Tech review updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Content modifications based on tech feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Content updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed unnecessary link.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed markdown comments.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Changed processor name references to remove tic marks.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits to fix links.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed common metrics from Metrics section and updated default value for .

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Updated OpenSearch security link.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed unnecessary column in Metrics table.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Minor update to title for accuracy.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed optional from each item in the Configuration table. All items are optional.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated links for OS security and service.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Updates made based on editorial feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update to phrasing.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-25 15:58:10 -06:00
Caroline ec668fdcba
Add OTel logs source doc to doc website. (#3608)
* Add otel_logs_source doc.

Signed-off-by: carolxob <carolxob@amazon.com>

* Content updates mainly to formatting.

Signed-off-by: carolxob <carolxob@amazon.com>

* Converted list entries into tables and fixed formatting.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adjusted nav_order in metadata.

Signed-off-by: carolxob <carolxob@amazon.com>

* Content edits from tech review.

Signed-off-by: carolxob <carolxob@amazon.com>

* Edit made based on technical review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Major revisions to table formatting and headers.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustements.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adjusted default values for items in the SSL table.

Signed-off-by: carolxob <carolxob@amazon.com>

* Reworded two options in the SSL section.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Minor changes.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor changes.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Content updates from feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor change.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Minor updates from editorial feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-25 15:57:32 -06:00
Caroline abd544826b
Add OTel metrics raw processor doc to doc website. (#3235)
* Opening new PR for file.

Signed-off-by: carolxob <carolxob@amazon.com>

* Add otel metrics raw processor doc.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor title change.

Signed-off-by: carolxob <carolxob@amazon.com>

* Delete mutate-event.md

Removing file - it is not part of this PR.

* Minor edits to format data as table.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates based on technical feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to common metrics section.

Signed-off-by: carolxob <carolxob@amazon.com>

* Incorporated technical feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor rephrasing.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update otel-metrics-raw.md

Fixed typo.

* Updated phrasing.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics-raw.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics-raw.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics-raw.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics-raw.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics-raw.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics-raw.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics-raw.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics-raw.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updated

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to metadata and first heading to ensure page displays correctly.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated title.

Signed-off-by: carolxob <carolxob@amazon.com>

* Renamed file for consistency with processor title.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Minor update.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor changes.

Signed-off-by: carolxob <carolxob@amazon.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-25 15:50:41 -06:00
David Venable aa2033c863
Updates the Data Prepper S3 source documentation (#3813)
* Updates the Data Prepper S3 source documentation with additional instructions for usage and new features in 2.2. Includes some clean-up as well.

Signed-off-by: David Venable <dlv@amazon.com>

* Add back in example

* Update _data-prepper/pipelines/configuration/sources/s3.md

Co-authored-by: Caroline <113052567+carolxob@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/s3.md

Co-authored-by: Caroline <113052567+carolxob@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/s3.md

Co-authored-by: Caroline <113052567+carolxob@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/s3.md

Co-authored-by: Caroline <113052567+carolxob@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update s3.md

* Apply suggestions from code review

Co-authored-by: Caroline <113052567+carolxob@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/s3.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update s3.md

Implement Caroline's feedback

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/s3.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/s3.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/s3.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/s3.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: David Venable <dlv@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Caroline <113052567+carolxob@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-25 13:10:40 -05:00
Liz Snyder 6202845539
Add max_retries option (#3835)
Signed-off-by: Liz Snyder <elizabsn@amazon.com>
2023-04-25 10:40:43 -05:00
Caroline 11c3e98335
Add examples to Key value page (#3105)
* Created PR to add examples to Key value doc.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustements.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update to table formatting.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor change.

Signed-off-by: carolxob <carolxob@amazon.com>

* Corrected table formatting.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added examples field table data.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor phrasing change.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated regex definition.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to intro.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to intro.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added default value column.?

Signed-off-by: carolxob <carolxob@amazon.com>

* Edits based on doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor changes.

Signed-off-by: carolxob <carolxob@amazon.com>

* Incorporated doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Major modifications to table formatting.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates to phrasing for examples for clarity.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Minor update to content.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>
Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-25 09:07:10 -06:00
Naarcha-AWS e1a1f44dd6
Add list to map processor (#3806)
* Add list to map processor.

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Tweak one last file

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Fix typo

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

Co-authored-by: Hai Yan <8153134+oeyh@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

Co-authored-by: Hai Yan <8153134+oeyh@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

Co-authored-by: Hai Yan <8153134+oeyh@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

Co-authored-by: Hai Yan <8153134+oeyh@users.noreply.github.com>

* Update mutate-event.md

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

* Add Chris' feedback

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* A couple more wording tweaks

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>

---------

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
Co-authored-by: Hai Yan <8153134+oeyh@users.noreply.github.com>
Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-20 13:48:36 -05:00
Liz Snyder 586cb3d2ff
Add opensearch sink security (#3809)
* Add opensearch sink security

Signed-off-by: Liz Snyder <elizabsn@amazon.com>

* Tweaks

Signed-off-by: Liz Snyder <elizabsn@amazon.com>

* Fix trust policy

Signed-off-by: Liz Snyder <elizabsn@amazon.com>

* Fix resource policy

Signed-off-by: Liz Snyder <elizabsn@amazon.com>

* Update _data-prepper/pipelines/configuration/sinks/opensearch.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/sinks/opensearch.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/sinks/opensearch.md

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Additional editorial feedback

* Make opensearch sink use consistent

* Last piece of editorial feedback

---------

Signed-off-by: Liz Snyder <elizabsn@amazon.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-20 13:47:54 -05:00
Naarcha-AWS 098f6d961a
Add E2E acknowledgements (#3811)
* Add E2E acknowledgements

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Fix typos

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Remove E from achknowledgments

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* another e

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Fix link

* Adjust S3 source.

* Update _data-prepper/pipelines/configuration/sources/s3.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/sources/s3.md

* Update _data-prepper/pipelines/configuration/sources/s3.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/pipelines.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/pipelines.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/pipelines.md

---------

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-20 12:39:24 -05:00
Caroline 8de7c3e3d3
Add Anomaly detector plugin doc. (#3569)
* Initial commit for new plugin doc.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to content.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added questions to get technical review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to phrasing.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed markdown comment.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed markdown comments.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed markdown comments and added in GitHub.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed markdown comments.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed markdown comments, added table header.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to table.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to table.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates to random_cut_forest mode section.

Signed-off-by: carolxob <carolxob@amazon.com>

* Content edits from tech review feedback

Signed-off-by: carolxob <carolxob@amazon.com>

* Content modifications based on tech feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Content edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits to bookmark/link.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor change to link.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update to content from tech feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed a 'this may appear in future releases' sentence.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Minor edits from editorial feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update.

Signed-off-by: carolxob <carolxob@amazon.com>

* Phrasing changes.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor change.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor phrasing change.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor phrasing update to RCF reference.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>
Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-20 10:30:22 -06:00
Caroline 8eac00ec8f
Add Trace peer forwarder doc. (#3238)
* Add trace peer forwarder doc.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated file title.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates based on tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Fixed header info and links.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added questions as comments to ask SME.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added Trace analytics link.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed markdown comments and added them in GitHub.

Signed-off-by: carolxob <carolxob@amazon.com>

* Made updates based on tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed markdown comment.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor change to unnecessary punctuation.

Signed-off-by: carolxob <carolxob@amazon.com>

* Some feedback from doc review.

Signed-off-by: carolxob <carolxob@amazon.com>

* Fixed broken links.

Signed-off-by: carolxob <carolxob@amazon.com>

* Fixed links.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates based on editorial feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/trace-peer-forwarder.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/trace-peer-forwarder.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Fixed hyperlinks to include data-prepper in path for peer forwarder.

Signed-off-by: carolxob <carolxob@amazon.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-11 11:35:19 -06:00
Caroline a246c60102
Add Parse JSON processor doc. (#3237)
* Add Parse JSON processor doc.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Formatted some information as a table instead of a list.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed smaller Parse JSON file.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits to table.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update based on tech feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Modified 'indices' to 'indexes'

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Modified title.

Signed-off-by: carolxob <carolxob@amazon.com>

* Content edits from doc and editorial reviews.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Final review comments incorporated.

Signed-off-by: carolxob <carolxob@amazon.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
2023-04-06 10:49:18 -06:00
Daniel (dB.) Doubrovkine 680c821937
Add CI with link checker. (#3584)
* Add CI with link checker.

Signed-off-by: dblock <dblock@amazon.com>

* Capture URI::InvalidURIError.

Signed-off-by: dblock <dblock@amazon.com>

* Use HEAD and catch URI errors.

Signed-off-by: dblock <dblock@amazon.com>

* Retry on a 405 with a GET.

Signed-off-by: dblock <dblock@amazon.com>

* Replaced external link checker with ruby-link-checker.

Signed-off-by: dblock <dblock@amazon.com>

* Don't exit with an exception.

Signed-off-by: dblock <dblock@amazon.com>

* Run internal link checker on build/ci.

Signed-off-by: dblock <dblock@amazon.com>

* Added broken links issue template.

Signed-off-by: dblock <dblock@amazon.com>

* Added host exclusions that 404 or fail on bots.

Signed-off-by: dblock <dblock@amazon.com>

* Raise anyway because Jekyll does it for us.

Signed-off-by: dblock <dblock@amazon.com>

* Fix broken links.

Signed-off-by: dblock <dblock@amazon.com>

* Only run link checker on main.

Signed-off-by: dblock <dblock@amazon.com>

* Re-add check-links.sh.

Signed-off-by: dblock <dblock@amazon.com>

* Run once a day on cron.

Signed-off-by: dblock <dblock@amazon.com>

---------

Signed-off-by: dblock <dblock@amazon.com>
2023-04-04 15:49:27 -04:00