Commit Graph

48 Commits

Author SHA1 Message Date
David Venable 27cbf9d03a
Removes the acknowledgements_timeout configuration from the Data Prepper Kafka source. (#4898)
Signed-off-by: David Venable <dlv@amazon.com>
2023-08-28 11:47:36 -05:00
David Venable 000aecc1a3
Moves the generic sink options into the Sinks documentation rather than PIpeline Configuration options. (#4907)
Signed-off-by: David Venable <dlv@amazon.com>
2023-08-28 10:43:26 -05:00
David Venable 5306134c41
Updated the intro text for the s3 sink. (#4854)
Signed-off-by: David Venable <dlv@amazon.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
2023-08-23 13:33:46 -05:00
Naarcha-AWS 1250649984
Fix typos on S3 sink documentation (#4862)
* Fix typos on S3 sink documentation

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sinks/s3.md

Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>
2023-08-23 12:49:46 -05:00
David Venable ed21ec74cf
Removes the Data Prepper csv sink codec. This is not ready for use and should not be documented. (#4859)
Signed-off-by: David Venable <dlv@amazon.com>
2023-08-23 09:19:04 -05:00
Asif Sohail Mohammed cc21df7ee8
S3 scan doc update (#4848)
* Updated s3 scan docs

Signed-off-by: Asif Sohail Mohammed <nsifmoh@amazon.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Asif Sohail Mohammed <nsifmoh@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
2023-08-22 15:28:13 -05:00
David Venable 0b5011409a
Data Prepper documentation updates for the s3 sink in 2.4.0 (#4847)
* Data Prepper documentation updates for the s3 sink in 2.4.0. Includes other corrections.

* Updated the Parquet/Avro codecs to include auto_schema as well as some clarifications on the restrictions of the auto-generated schema.

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
2023-08-22 15:18:40 -05:00
Asif Sohail Mohammed 8e9d0ed4de
S3 scan docs (#4756)
* Add notification_source option for S3

Signed-off-by: Asif Sohail Mohammed <nsifmoh@amazon.com>

* Added S3 scan configuration

Signed-off-by: Asif Sohail Mohammed <nsifmoh@amazon.com>

* Addressed some failing style checks

Signed-off-by: Asif Sohail Mohammed <nsifmoh@amazon.com>

* Edit S3 additions.

* Fix links.

* Fix style flaw

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update s3.md

---------

Signed-off-by: Asif Sohail Mohammed <nsifmoh@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-08-21 11:02:07 -05:00
David Venable ac3f60d5ca
Documentation for bucket ownership features in Data Prepper 2.4 (#4679)
* Adds documentation for new features in Data Prepper 2.4 related to bucket ownership. Includes a section explaining how the bucket ownership works and can be configured for cross-account access. Resolves #4678

Signed-off-by: David Venable <dlv@amazon.com>

* Updates from PR feedback.

Signed-off-by: David Venable <dlv@amazon.com>

* Remove redundant header

Add a name to primary H2 header to section to prevent links from breaking.

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: David Venable <dlv@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-08-21 10:46:04 -05:00
Qi Chen 7d077786bc
MAINT: update distribution_version in opensearch sink (#4742)
* MAINT: update distribution_version in opensearch sink

Signed-off-by: George Chen <qchea@amazon.com>

* Update _data-prepper/pipelines/configuration/sinks/opensearch.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Qi Chen <chenqi0805@gmail.com>

* Update _data-prepper/pipelines/configuration/sinks/opensearch.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Qi Chen <chenqi0805@gmail.com>

* Fix typo

* ADD: request compression flag

Signed-off-by: George Chen <qchea@amazon.com>

* Update opensearch.md

---------

Signed-off-by: George Chen <qchea@amazon.com>
Signed-off-by: Qi Chen <chenqi0805@gmail.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
2023-08-21 10:45:40 -05:00
Naarcha-AWS 152ffcd79f
Add Kafka source for DP 2.4 (#4737)
* Add Kafka source for DP 2.4

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Use new Kafka settings

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Add technical feedback

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Fix typos

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Apply suggestions from code review

Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM>
Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Change array to object.

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Last minute changes to Kafka defaults

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-08-21 10:43:32 -05:00
David Venable 442f4c446a
Documents general sink options (#4759)
* Adds general sink options which includes: routes, tags_target_key, include_keys, exclude_keys.

Signed-off-by: David Venable <dlv@amazon.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: David Venable <dlv@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
2023-08-21 10:39:31 -05:00
Naarcha-AWS c14b8b2a07
Add CIDR and User Agent processors (#4585)
* Add CIDR and User Agent processors

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Add CIDR functions

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Fix typos

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Add technical feedback

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Update user-agent.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/expression-syntax.md

Co-authored-by: Hai Yan <8153134+oeyh@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update user-agent.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>
Co-authored-by: Hai Yan <8153134+oeyh@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-07-27 14:58:47 -05:00
Naarcha-AWS 0fcfc4862e
Add S3 sink documentation (#4340)
* Add S3 sink documentation

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Update _data-prepper/pipelines/configuration/sinks/s3.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sinks/s3.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sinks/s3.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-07-18 16:06:06 -07:00
Naarcha-AWS adae164d53
Add obfuscate processor (#4515)
* Add obfuscate processor reference

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* fix typo

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Fix paragraph description.

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Fix consistency issues

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Apply suggestions from code review

Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-07-14 12:44:01 -05:00
Powershellfish d78bd5b70e
Update capitalization for Events (#4213)
Signed-off-by: Powershellfish <47839113+powershellfish@users.noreply.github.com>
2023-07-05 13:50:27 -05:00
David Venable 5a4daa4225
Documents the Data Prepper opensearch sink's template_type parameter. (#4290)
* Documents the Data Prepper opensearch sink's template_type parameter.

Signed-off-by: David Venable <dlv@amazon.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: David Venable <dlv@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
2023-06-13 13:53:29 -05:00
Powershellfish 3b9ae603de
Updated capitalization. (#4214)
Signed-off-by: Powershellfish <47839113+powershellfish@users.noreply.github.com>
2023-06-07 06:41:42 -07:00
Powershellfish baa2fb45ef
[DOC] Remove indentation in copy_values processor (#4057)
* Test.

Signed-off-by: Powershellfish <47839113+powershellfish@users.noreply.github.com>

* Update index.md

Testing ability to create PRs.

* Removed indentation.

Signed-off-by: Powershellfish <47839113+powershellfish@users.noreply.github.com>

---------

Signed-off-by: Powershellfish <47839113+powershellfish@users.noreply.github.com>
2023-05-15 17:49:13 -07:00
kolchfa-aws 8463c8f278
Correct plugin capitalization (#3838)
* Correct plugin capitalization

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Revert cluster-stats because the name is in response

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Revert cluster-stats once more

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

---------

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>
2023-05-04 11:11:54 -04:00
Heather Halter f35e72d577
fix list-to-map order (#3941)
Signed-off-by: Heather Halter <hdhalter@amazon.com>
2023-05-03 10:37:27 -07:00
Heather Halter 0235af562f
fix table headings (#3943)
Signed-off-by: Heather Halter <hdhalter@amazon.com>
2023-05-03 07:04:30 -07:00
Naarcha-AWS 1259454b73
Fix typo on processor page. (#3903)
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
2023-04-27 18:58:54 -05:00
Naarcha-AWS 4251fb04bd
Add DLQ and aggregate processor edits. Baseline processor names (#3880)
* Add DLQ and aggregate processor edits. Baseline processor names

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Remove processor from titles

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Update dlq.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Add additional feedback

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* add cross links

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Fix typos

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Add doc review

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Remove link

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Apply suggestions from code review

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update dlq.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Fix processor title

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-27 14:01:01 -05:00
Caroline 66f30d757f
Add DLQ to OpenSearch sink configuration options. (#3743)
* Add DLQ to OpenSearch sink configuration options.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/sinks/opensearch.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
2023-04-26 11:36:47 -05:00
Caroline ab10e6a296
Add OTel trace group processor to doc website. (#3236)
* Add otel trace group processor.

Signed-off-by: carolxob <carolxob@amazon.com>

* Copy edits to improve table formatting.

Signed-off-by: carolxob <carolxob@amazon.com>

* Incorporated doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Tech review updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Content modifications based on tech feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Content updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed unnecessary link.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed markdown comments.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Changed processor name references to remove tic marks.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits to fix links.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed common metrics from Metrics section and updated default value for .

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Updated OpenSearch security link.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed unnecessary column in Metrics table.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Minor update to title for accuracy.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed optional from each item in the Configuration table. All items are optional.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated links for OS security and service.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Updates made based on editorial feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update to phrasing.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-trace-group.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-25 15:58:10 -06:00
Caroline ec668fdcba
Add OTel logs source doc to doc website. (#3608)
* Add otel_logs_source doc.

Signed-off-by: carolxob <carolxob@amazon.com>

* Content updates mainly to formatting.

Signed-off-by: carolxob <carolxob@amazon.com>

* Converted list entries into tables and fixed formatting.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adjusted nav_order in metadata.

Signed-off-by: carolxob <carolxob@amazon.com>

* Content edits from tech review.

Signed-off-by: carolxob <carolxob@amazon.com>

* Edit made based on technical review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Major revisions to table formatting and headers.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustements.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adjusted default values for items in the SSL table.

Signed-off-by: carolxob <carolxob@amazon.com>

* Reworded two options in the SSL section.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Minor changes.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor changes.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Content updates from feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor change.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Minor updates from editorial feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-25 15:57:32 -06:00
Caroline abd544826b
Add OTel metrics raw processor doc to doc website. (#3235)
* Opening new PR for file.

Signed-off-by: carolxob <carolxob@amazon.com>

* Add otel metrics raw processor doc.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor title change.

Signed-off-by: carolxob <carolxob@amazon.com>

* Delete mutate-event.md

Removing file - it is not part of this PR.

* Minor edits to format data as table.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates based on technical feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to common metrics section.

Signed-off-by: carolxob <carolxob@amazon.com>

* Incorporated technical feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor rephrasing.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update otel-metrics-raw.md

Fixed typo.

* Updated phrasing.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics-raw.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics-raw.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics-raw.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics-raw.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics-raw.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics-raw.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics-raw.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics-raw.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updated

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to metadata and first heading to ensure page displays correctly.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated title.

Signed-off-by: carolxob <carolxob@amazon.com>

* Renamed file for consistency with processor title.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/otel-metrics.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Minor update.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor changes.

Signed-off-by: carolxob <carolxob@amazon.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-25 15:50:41 -06:00
David Venable aa2033c863
Updates the Data Prepper S3 source documentation (#3813)
* Updates the Data Prepper S3 source documentation with additional instructions for usage and new features in 2.2. Includes some clean-up as well.

Signed-off-by: David Venable <dlv@amazon.com>

* Add back in example

* Update _data-prepper/pipelines/configuration/sources/s3.md

Co-authored-by: Caroline <113052567+carolxob@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/s3.md

Co-authored-by: Caroline <113052567+carolxob@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/s3.md

Co-authored-by: Caroline <113052567+carolxob@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/s3.md

Co-authored-by: Caroline <113052567+carolxob@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update s3.md

* Apply suggestions from code review

Co-authored-by: Caroline <113052567+carolxob@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/s3.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update s3.md

Implement Caroline's feedback

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/s3.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/s3.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/s3.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/sources/s3.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: David Venable <dlv@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Caroline <113052567+carolxob@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-25 13:10:40 -05:00
Liz Snyder 6202845539
Add max_retries option (#3835)
Signed-off-by: Liz Snyder <elizabsn@amazon.com>
2023-04-25 10:40:43 -05:00
Caroline 11c3e98335
Add examples to Key value page (#3105)
* Created PR to add examples to Key value doc.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustements.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update to table formatting.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor change.

Signed-off-by: carolxob <carolxob@amazon.com>

* Corrected table formatting.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added examples field table data.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor phrasing change.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated regex definition.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to intro.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to intro.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added default value column.?

Signed-off-by: carolxob <carolxob@amazon.com>

* Edits based on doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor changes.

Signed-off-by: carolxob <carolxob@amazon.com>

* Incorporated doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Major modifications to table formatting.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates to phrasing for examples for clarity.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Minor update to content.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/key-value.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com>
Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-25 09:07:10 -06:00
Naarcha-AWS e1a1f44dd6
Add list to map processor (#3806)
* Add list to map processor.

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Tweak one last file

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Fix typo

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

Co-authored-by: Hai Yan <8153134+oeyh@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

Co-authored-by: Hai Yan <8153134+oeyh@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

Co-authored-by: Hai Yan <8153134+oeyh@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

Co-authored-by: Hai Yan <8153134+oeyh@users.noreply.github.com>

* Update mutate-event.md

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

* Add Chris' feedback

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* A couple more wording tweaks

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/list-to-map.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>

---------

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
Co-authored-by: Hai Yan <8153134+oeyh@users.noreply.github.com>
Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-20 13:48:36 -05:00
Liz Snyder 586cb3d2ff
Add opensearch sink security (#3809)
* Add opensearch sink security

Signed-off-by: Liz Snyder <elizabsn@amazon.com>

* Tweaks

Signed-off-by: Liz Snyder <elizabsn@amazon.com>

* Fix trust policy

Signed-off-by: Liz Snyder <elizabsn@amazon.com>

* Fix resource policy

Signed-off-by: Liz Snyder <elizabsn@amazon.com>

* Update _data-prepper/pipelines/configuration/sinks/opensearch.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/sinks/opensearch.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/sinks/opensearch.md

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Additional editorial feedback

* Make opensearch sink use consistent

* Last piece of editorial feedback

---------

Signed-off-by: Liz Snyder <elizabsn@amazon.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-20 13:47:54 -05:00
Naarcha-AWS 098f6d961a
Add E2E acknowledgements (#3811)
* Add E2E acknowledgements

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Fix typos

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Remove E from achknowledgments

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* another e

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Fix link

* Adjust S3 source.

* Update _data-prepper/pipelines/configuration/sources/s3.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/sources/s3.md

* Update _data-prepper/pipelines/configuration/sources/s3.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/pipelines.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/pipelines.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/pipelines.md

---------

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-20 12:39:24 -05:00
Caroline 8de7c3e3d3
Add Anomaly detector plugin doc. (#3569)
* Initial commit for new plugin doc.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to content.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added questions to get technical review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to phrasing.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed markdown comment.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed markdown comments.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed markdown comments and added in GitHub.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed markdown comments.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed markdown comments, added table header.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to table.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to table.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates to random_cut_forest mode section.

Signed-off-by: carolxob <carolxob@amazon.com>

* Content edits from tech review feedback

Signed-off-by: carolxob <carolxob@amazon.com>

* Content modifications based on tech feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Content edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits to bookmark/link.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor change to link.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update to content from tech feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed a 'this may appear in future releases' sentence.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Minor edits from editorial feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update.

Signed-off-by: carolxob <carolxob@amazon.com>

* Phrasing changes.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor change.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor phrasing change.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor phrasing update to RCF reference.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/anomaly-detector.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com>
Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-20 10:30:22 -06:00
Caroline 8eac00ec8f
Add Trace peer forwarder doc. (#3238)
* Add trace peer forwarder doc.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated file title.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates based on tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Fixed header info and links.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added questions as comments to ask SME.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added Trace analytics link.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed markdown comments and added them in GitHub.

Signed-off-by: carolxob <carolxob@amazon.com>

* Made updates based on tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed markdown comment.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor change to unnecessary punctuation.

Signed-off-by: carolxob <carolxob@amazon.com>

* Some feedback from doc review.

Signed-off-by: carolxob <carolxob@amazon.com>

* Fixed broken links.

Signed-off-by: carolxob <carolxob@amazon.com>

* Fixed links.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates based on editorial feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/trace-peer-forwarder.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/trace-peer-forwarder.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Fixed hyperlinks to include data-prepper in path for peer forwarder.

Signed-off-by: carolxob <carolxob@amazon.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-11 11:35:19 -06:00
Caroline a246c60102
Add Parse JSON processor doc. (#3237)
* Add Parse JSON processor doc.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Formatted some information as a table instead of a list.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed smaller Parse JSON file.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits to table.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update based on tech feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Modified 'indices' to 'indexes'

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Modified title.

Signed-off-by: carolxob <carolxob@amazon.com>

* Content edits from doc and editorial reviews.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Final review comments incorporated.

Signed-off-by: carolxob <carolxob@amazon.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
2023-04-06 10:49:18 -06:00
Daniel (dB.) Doubrovkine 680c821937
Add CI with link checker. (#3584)
* Add CI with link checker.

Signed-off-by: dblock <dblock@amazon.com>

* Capture URI::InvalidURIError.

Signed-off-by: dblock <dblock@amazon.com>

* Use HEAD and catch URI errors.

Signed-off-by: dblock <dblock@amazon.com>

* Retry on a 405 with a GET.

Signed-off-by: dblock <dblock@amazon.com>

* Replaced external link checker with ruby-link-checker.

Signed-off-by: dblock <dblock@amazon.com>

* Don't exit with an exception.

Signed-off-by: dblock <dblock@amazon.com>

* Run internal link checker on build/ci.

Signed-off-by: dblock <dblock@amazon.com>

* Added broken links issue template.

Signed-off-by: dblock <dblock@amazon.com>

* Added host exclusions that 404 or fail on bots.

Signed-off-by: dblock <dblock@amazon.com>

* Raise anyway because Jekyll does it for us.

Signed-off-by: dblock <dblock@amazon.com>

* Fix broken links.

Signed-off-by: dblock <dblock@amazon.com>

* Only run link checker on main.

Signed-off-by: dblock <dblock@amazon.com>

* Re-add check-links.sh.

Signed-off-by: dblock <dblock@amazon.com>

* Run once a day on cron.

Signed-off-by: dblock <dblock@amazon.com>

---------

Signed-off-by: dblock <dblock@amazon.com>
2023-04-04 15:49:27 -04:00
Caroline 04f12af4a4
Add Mutate event processor doc. (#3234)
* Opening new PR for file.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustements.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adjustements to tables.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed some markdown comments.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adjustements to tables.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates based on doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* MAde changes based on tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated text for YAML file path locations.

Signed-off-by: carolxob <carolxob@amazon.com>

* Edits made based on technical review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits from tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates based on tech review feedback and formatting.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor upate from doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Edits from doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates based on doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added copy button to relevant code blocks.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added links.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits based on doc review.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits based on doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Edits made based on doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update to include key examples.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Fixing commit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Incorporating editorial review changes.

Signed-off-by: carolxob <carolxob@amazon.com>

* Incorporated the rest of editorial feedback received.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates based on feedback comments.

Signed-off-by: carolxob <carolxob@amazon.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-03 14:52:04 -06:00
Caroline 78df4883b9
Fix table header in Bounded blocking doc. (#3450)
Signed-off-by: carolxob <carolxob@amazon.com>
2023-03-16 14:33:54 -05:00
Caroline 0c2c72fbe8
Fix table formatting for http_source doc. (#3451)
Signed-off-by: carolxob <carolxob@amazon.com>
2023-03-16 11:47:57 -05:00
Caroline 0690c47032
Add documentation for Mutate strings (#2950)
* Added placeholder files.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added source content from Data Prepper repo docs.

Signed-off-by: carolxob <carolxob@amazon.com>

* Edits for consistency and formatting.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits to otel-metrics-string.

Signed-off-by: carolxob <carolxob@amazon.com>

* Content and formatting edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits to trace-peer-forwarder

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated links to existing Data Prepper documentation as relative URLs.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits for consistency and clarity.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update.

Signed-off-by: carolxob <carolxob@amazon.com>

* minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adjustments based on technical feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed 'Developer guide' section.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustments for clarity.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustments for consistency.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed markdown comment.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates made based on doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed bullet for single item list.

Signed-off-by: carolxob <carolxob@amazon.com>

* Edits made based on doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Modified file title and doc title to more accurately reflect contents.

Signed-off-by: carolxob <carolxob@amazon.com>

* Incorporated feedback from tech review.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added note to convert to table.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated links.

Signed-off-by: carolxob <carolxob@amazon.com>

* Incorporated most doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed unaffected files.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustements.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added some clarification around the substitute string processor.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adjusted table boundaries.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update based on tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed Basic from Usage section titles.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added link placeholder for config file instructions.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update with sentence structure.

Signed-off-by: carolxob <carolxob@amazon.com>

* Refined steps around Data Prepper configuration files.

Signed-off-by: carolxob <carolxob@amazon.com>

* Renamed file for consistency.

Signed-off-by: carolxob <carolxob@amazon.com>

* Changed phrasing for consistency.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed redundant processor docs.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates from doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Fixed header justification alignment.

Signed-off-by: carolxob <carolxob@amazon.com>

* Made edits based on editorial review.

Signed-off-by: carolxob <carolxob@amazon.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
2023-03-16 10:28:54 -05:00
Caroline ef83f6d7d5
Added metrics sections to Aggregate processor subpages. (#2730)
* Added metrics section to Aggregate processor page.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added Metrics section to individual Processors pages.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added metrics section for JSON processor.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added metrics sections. Changed Default is to Default value is.

Signed-off-by: carolxob <carolxob@amazon.com>

* Corrected references from AWS S3 to Amazon S3.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to Metrics sections and phrasing.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated Action link.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates based on tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates based on tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to buffer_size and batch_size default values.

Signed-off-by: carolxob <carolxob@amazon.com>

* Edits to Metrics sections for each processor.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update made based ondoc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to intro text for processor pages. Minor adjustements to other text for clarity.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adjustements to phrasing, fixed typos.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to word choice and corrected a typo.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Made updates based ondoc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates to http-source.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added common processors table to affected docs.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update to one file.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update based on tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Major editorial feedback incorporated through key-value.md.

Signed-off-by: carolxob <carolxob@amazon.com>

* Incorporated major editorial feedback thup to service-map-stateful.

Signed-off-by: carolxob <carolxob@amazon.com>

* Incorporated major editorial feedback for Processors section.

Signed-off-by: carolxob <carolxob@amazon.com>

* Major editorial updates, specifically to inclusion of text introducing option configuration tables.

Signed-off-by: carolxob <carolxob@amazon.com>

* Major editorial feedback through otel-trace.md incorporated.

Signed-off-by: carolxob <carolxob@amazon.com>

* Major editorial edits incorporated.

Signed-off-by: carolxob <carolxob@amazon.com>

* Technical feedback and editorial feedback incorporated.

Signed-off-by: carolxob <carolxob@amazon.com>

* Incorporated missing editorial feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustements to OpenSearch sink.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor changes to capitalization.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Made one instance of processor name consistent with other references.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update based on editorial feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
2023-02-27 11:40:59 -05:00
Caroline 3ae22302a2
Updated authentication links. (#2812)
Signed-off-by: carolxob <carolxob@amazon.com>
2023-02-10 11:53:58 -07:00
JannikBrand 6ccd78c4fb
Update data-prepper's default buffer values (#2574)
Since data-prepper version 2.0.0 there are new default values for buffer_size & batch_size.
The default buffer_size (previously: 512) and default batch_size (previously: 8) were increased by the factor 25.
The new default values are: buffer_size: 12800 and batch_size: 200.

Issue: https://github.com/opensearch-project/data-prepper/issues/1983

Signed-off-by: Jannik Brand <jannik.brand@sap.com>
2023-02-09 10:44:33 -06:00
Caroline 5a9e2f7748
Removed duplicate content after introduction. It is contained in the child pages for this section. (#2727)
Signed-off-by: carolxob <carolxob@amazon.com>
2023-02-08 14:44:38 -07:00
Caroline 6791282a12
Commented out a section that will be built out later. (#2721)
Signed-off-by: carolxob <carolxob@amazon.com>
2023-02-08 14:43:52 -07:00
Caroline 0249991f76
Data Prepper ToC Update (#2514)
* Creating PR with first file.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adding newly created files to PR.

Signed-off-by: carolxob <carolxob@amazon.com>

* Reorganized files and added appropriate metadata to map ToC correctly.

Signed-off-by: carolxob <carolxob@amazon.com>

* Moved Authoring pipelines page.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor ToC updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor ToC updates to Sources section for Data Prepper.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated Buffers section under Data Prepper.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update to otelmetricssource.

Signed-off-by: carolxob <carolxob@amazon.com>

* Restructured ToC in Processors section for Data Prepper.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor filename change.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adjustments to metadata in ToC.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Fixed nav order in metadata.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update top metadata for ToC.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adjustmenets to Toc order.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustments to ToC metadata.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustments to Sinks section.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adjustements to high level ToC.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustement to Pipelines.md

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update.

Signed-off-by: carolxob <carolxob@amazon.com>

* Slight reorganization. Removed two placeholder pages for now.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed a page and replaced with pipelines content.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor changes/additions to content for placeholder pages.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update to page link.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustments to ToC metadata.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed /clients from redirects to correct nav order.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustments to ToC metadata.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustments.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustment ot metadata.

Signed-off-by: carolxob <carolxob@amazon.com>

* TOC link fixes

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Changed page name.

Signed-off-by: carolxob <carolxob@amazon.com>

* Corrected references to Peer Forwarder.

Signed-off-by: carolxob <carolxob@amazon.com>

* Renamed Data Prepper folder.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to phrasing and capitalization.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor phrasing update.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor phrasing update.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor change.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor change to change S3 Source to S3Source.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated references to peer forwarder and changed capitalization.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated capitalization for peer forwarder.

Signed-off-by: carolxob <carolxob@amazon.com>

* Made edits based on doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update to one word.

Signed-off-by: carolxob <carolxob@amazon.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
Co-authored-by: Naarcha-AWS <naarcha@amazon.com>
2023-02-03 15:06:10 -07:00