Commit Graph

62 Commits

Author SHA1 Message Date
Caroline a246c60102
Add Parse JSON processor doc. (#3237)
* Add Parse JSON processor doc.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Formatted some information as a table instead of a list.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed smaller Parse JSON file.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits to table.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update based on tech feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Modified 'indices' to 'indexes'

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Modified title.

Signed-off-by: carolxob <carolxob@amazon.com>

* Content edits from doc and editorial reviews.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Final review comments incorporated.

Signed-off-by: carolxob <carolxob@amazon.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
2023-04-06 10:49:18 -06:00
Daniel (dB.) Doubrovkine 680c821937
Add CI with link checker. (#3584)
* Add CI with link checker.

Signed-off-by: dblock <dblock@amazon.com>

* Capture URI::InvalidURIError.

Signed-off-by: dblock <dblock@amazon.com>

* Use HEAD and catch URI errors.

Signed-off-by: dblock <dblock@amazon.com>

* Retry on a 405 with a GET.

Signed-off-by: dblock <dblock@amazon.com>

* Replaced external link checker with ruby-link-checker.

Signed-off-by: dblock <dblock@amazon.com>

* Don't exit with an exception.

Signed-off-by: dblock <dblock@amazon.com>

* Run internal link checker on build/ci.

Signed-off-by: dblock <dblock@amazon.com>

* Added broken links issue template.

Signed-off-by: dblock <dblock@amazon.com>

* Added host exclusions that 404 or fail on bots.

Signed-off-by: dblock <dblock@amazon.com>

* Raise anyway because Jekyll does it for us.

Signed-off-by: dblock <dblock@amazon.com>

* Fix broken links.

Signed-off-by: dblock <dblock@amazon.com>

* Only run link checker on main.

Signed-off-by: dblock <dblock@amazon.com>

* Re-add check-links.sh.

Signed-off-by: dblock <dblock@amazon.com>

* Run once a day on cron.

Signed-off-by: dblock <dblock@amazon.com>

---------

Signed-off-by: dblock <dblock@amazon.com>
2023-04-04 15:49:27 -04:00
Caroline 04f12af4a4
Add Mutate event processor doc. (#3234)
* Opening new PR for file.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustements.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adjustements to tables.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed some markdown comments.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adjustements to tables.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates based on doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* MAde changes based on tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated text for YAML file path locations.

Signed-off-by: carolxob <carolxob@amazon.com>

* Edits made based on technical review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits from tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates based on tech review feedback and formatting.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor upate from doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Edits from doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates based on doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added copy button to relevant code blocks.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added links.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits based on doc review.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits based on doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Edits made based on doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update to include key examples.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Fixing commit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Update _data-prepper/pipelines/configuration/processors/mutate-event.md

Co-authored-by: Nathan Bower <nbower@amazon.com>

* Incorporating editorial review changes.

Signed-off-by: carolxob <carolxob@amazon.com>

* Incorporated the rest of editorial feedback received.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates based on feedback comments.

Signed-off-by: carolxob <carolxob@amazon.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-03 14:52:04 -06:00
Caroline 78df4883b9
Fix table header in Bounded blocking doc. (#3450)
Signed-off-by: carolxob <carolxob@amazon.com>
2023-03-16 14:33:54 -05:00
Caroline 0c2c72fbe8
Fix table formatting for http_source doc. (#3451)
Signed-off-by: carolxob <carolxob@amazon.com>
2023-03-16 11:47:57 -05:00
Caroline 0690c47032
Add documentation for Mutate strings (#2950)
* Added placeholder files.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added source content from Data Prepper repo docs.

Signed-off-by: carolxob <carolxob@amazon.com>

* Edits for consistency and formatting.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits to otel-metrics-string.

Signed-off-by: carolxob <carolxob@amazon.com>

* Content and formatting edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits to trace-peer-forwarder

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated links to existing Data Prepper documentation as relative URLs.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits for consistency and clarity.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update.

Signed-off-by: carolxob <carolxob@amazon.com>

* minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adjustments based on technical feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed 'Developer guide' section.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustments for clarity.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustments for consistency.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed markdown comment.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates made based on doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed bullet for single item list.

Signed-off-by: carolxob <carolxob@amazon.com>

* Edits made based on doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Modified file title and doc title to more accurately reflect contents.

Signed-off-by: carolxob <carolxob@amazon.com>

* Incorporated feedback from tech review.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added note to convert to table.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated links.

Signed-off-by: carolxob <carolxob@amazon.com>

* Incorporated most doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed unaffected files.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustements.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added some clarification around the substitute string processor.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adjusted table boundaries.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update based on tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed Basic from Usage section titles.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added link placeholder for config file instructions.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update with sentence structure.

Signed-off-by: carolxob <carolxob@amazon.com>

* Refined steps around Data Prepper configuration files.

Signed-off-by: carolxob <carolxob@amazon.com>

* Renamed file for consistency.

Signed-off-by: carolxob <carolxob@amazon.com>

* Changed phrasing for consistency.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed redundant processor docs.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates from doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Fixed header justification alignment.

Signed-off-by: carolxob <carolxob@amazon.com>

* Made edits based on editorial review.

Signed-off-by: carolxob <carolxob@amazon.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
2023-03-16 10:28:54 -05:00
Caroline ef83f6d7d5
Added metrics sections to Aggregate processor subpages. (#2730)
* Added metrics section to Aggregate processor page.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added Metrics section to individual Processors pages.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added metrics section for JSON processor.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added metrics sections. Changed Default is to Default value is.

Signed-off-by: carolxob <carolxob@amazon.com>

* Corrected references from AWS S3 to Amazon S3.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to Metrics sections and phrasing.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated Action link.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates based on tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates based on tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to buffer_size and batch_size default values.

Signed-off-by: carolxob <carolxob@amazon.com>

* Edits to Metrics sections for each processor.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update made based ondoc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to intro text for processor pages. Minor adjustements to other text for clarity.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adjustements to phrasing, fixed typos.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to word choice and corrected a typo.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Made updates based ondoc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates to http-source.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added common processors table to affected docs.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update to one file.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update based on tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Major editorial feedback incorporated through key-value.md.

Signed-off-by: carolxob <carolxob@amazon.com>

* Incorporated major editorial feedback thup to service-map-stateful.

Signed-off-by: carolxob <carolxob@amazon.com>

* Incorporated major editorial feedback for Processors section.

Signed-off-by: carolxob <carolxob@amazon.com>

* Major editorial updates, specifically to inclusion of text introducing option configuration tables.

Signed-off-by: carolxob <carolxob@amazon.com>

* Major editorial feedback through otel-trace.md incorporated.

Signed-off-by: carolxob <carolxob@amazon.com>

* Major editorial edits incorporated.

Signed-off-by: carolxob <carolxob@amazon.com>

* Technical feedback and editorial feedback incorporated.

Signed-off-by: carolxob <carolxob@amazon.com>

* Incorporated missing editorial feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustements to OpenSearch sink.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor changes to capitalization.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Made one instance of processor name consistent with other references.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update based on editorial feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
2023-02-27 11:40:59 -05:00
Caroline 3ae22302a2
Updated authentication links. (#2812)
Signed-off-by: carolxob <carolxob@amazon.com>
2023-02-10 11:53:58 -07:00
JannikBrand 6ccd78c4fb
Update data-prepper's default buffer values (#2574)
Since data-prepper version 2.0.0 there are new default values for buffer_size & batch_size.
The default buffer_size (previously: 512) and default batch_size (previously: 8) were increased by the factor 25.
The new default values are: buffer_size: 12800 and batch_size: 200.

Issue: https://github.com/opensearch-project/data-prepper/issues/1983

Signed-off-by: Jannik Brand <jannik.brand@sap.com>
2023-02-09 10:44:33 -06:00
Caroline 5a9e2f7748
Removed duplicate content after introduction. It is contained in the child pages for this section. (#2727)
Signed-off-by: carolxob <carolxob@amazon.com>
2023-02-08 14:44:38 -07:00
Caroline 6791282a12
Commented out a section that will be built out later. (#2721)
Signed-off-by: carolxob <carolxob@amazon.com>
2023-02-08 14:43:52 -07:00
Caroline 0249991f76
Data Prepper ToC Update (#2514)
* Creating PR with first file.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adding newly created files to PR.

Signed-off-by: carolxob <carolxob@amazon.com>

* Reorganized files and added appropriate metadata to map ToC correctly.

Signed-off-by: carolxob <carolxob@amazon.com>

* Moved Authoring pipelines page.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor ToC updates.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor ToC updates to Sources section for Data Prepper.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated Buffers section under Data Prepper.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update to otelmetricssource.

Signed-off-by: carolxob <carolxob@amazon.com>

* Restructured ToC in Processors section for Data Prepper.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor filename change.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adjustments to metadata in ToC.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Fixed nav order in metadata.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update top metadata for ToC.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adjustmenets to Toc order.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustments to ToC metadata.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustments to Sinks section.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adjustements to high level ToC.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustement to Pipelines.md

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update.

Signed-off-by: carolxob <carolxob@amazon.com>

* Slight reorganization. Removed two placeholder pages for now.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed a page and replaced with pipelines content.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor changes/additions to content for placeholder pages.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update to page link.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustments to ToC metadata.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Removed /clients from redirects to correct nav order.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustments to ToC metadata.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustments.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustment ot metadata.

Signed-off-by: carolxob <carolxob@amazon.com>

* TOC link fixes

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Changed page name.

Signed-off-by: carolxob <carolxob@amazon.com>

* Corrected references to Peer Forwarder.

Signed-off-by: carolxob <carolxob@amazon.com>

* Renamed Data Prepper folder.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to phrasing and capitalization.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor phrasing update.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor phrasing update.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor change.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor change to change S3 Source to S3Source.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated references to peer forwarder and changed capitalization.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated capitalization for peer forwarder.

Signed-off-by: carolxob <carolxob@amazon.com>

* Made edits based on doc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update to one word.

Signed-off-by: carolxob <carolxob@amazon.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
Co-authored-by: Naarcha-AWS <naarcha@amazon.com>
2023-02-03 15:06:10 -07:00