Commit Graph

540 Commits

Author SHA1 Message Date
Sebastian Bazley 70f00f973a Fix readLine() so can still detect start of line next time round
Also fix broken test: format was set to ignore empty lines, yet it checked for one

git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306649 13f79535-47bb-0310-9956-ffa450edef68
2012-03-29 00:02:35 +00:00
Sebastian Bazley 7addc8f688 Misplaced comment
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306643 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 23:35:09 +00:00
Sebastian Bazley 8cd4024e6d Add experimental state-driven lexer
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306519 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 18:12:08 +00:00
Sebastian Bazley 6ca0f78156 Allow for new COMMENT token type
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306516 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 17:56:53 +00:00
Sebastian Bazley 61b66220b5 Add a trailing comment
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306432 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 15:35:10 +00:00
Sebastian Bazley 22af43218a Don't detect comments except at start of a line
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306422 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 15:31:20 +00:00
Sebastian Bazley a708256b65 Make lines clearer
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306414 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 15:27:53 +00:00
Sebastian Bazley 67390ea304 Handle CommentStart option
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306345 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 14:06:01 +00:00
Sebastian Bazley fcfb6dc7c5 CSV-84 Clarify comment handling
Test was wrong; line comment should have been dropped entirely.
Fix record handling accordingly

git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306342 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 14:05:25 +00:00
Sebastian Bazley e922bf949a CSV-84 Clarify comment handling
Fix code so comment only detected at start of a line

git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306325 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 13:46:27 +00:00
Sebastian Bazley 05b5c8ef48 Revert r1306321 - committed wrong file
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306324 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 13:44:38 +00:00
Sebastian Bazley 0fee01aa2d Add method for detecting start of line
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306322 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 13:41:42 +00:00
Sebastian Bazley 93089b260c Add method for detecting start of line
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306321 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 13:40:46 +00:00
Sebastian Bazley 6a91c05061 CSV-84 Clarify comment handling
Document how comment behaves

git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306316 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 13:35:38 +00:00
Sebastian Bazley f299569070 CSV-84 Clarify comment handling
Fix test case which expected to handle a trailing comment

git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306315 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 13:34:21 +00:00
Sebastian Bazley 0fb414b6f2 Give a bit more detail when reporting an error
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306313 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 13:32:14 +00:00
Sebastian Bazley c510c7222f Tidy up; rename shared fields to be clearer
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306307 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 13:18:11 +00:00
Sebastian Bazley ba26844c7b CSV-70 Improve readability of CSVLexer
Simplify; remove while(true) loop

git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306079 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 00:34:27 +00:00
Sebastian Bazley 7c2cfeaf58 lastChar cannot possibly be EOF if it is CR, LF or UNDEFINED
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306073 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 00:22:25 +00:00
Sebastian Bazley 0a0f3bbd36 CSV-70 Improve readability of CSVLexer
Introduce COMMENT type

git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306064 13f79535-47bb-0310-9956-ffa450edef68
2012-03-28 00:01:43 +00:00
Sebastian Bazley 9114337075 CSV-70 Improve readability of CSVLexer
Remove unnecessary parameters

git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306062 13f79535-47bb-0310-9956-ffa450edef68
2012-03-27 23:50:04 +00:00
Sebastian Bazley fba580930f Provide toString method for IDE debugging
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306047 13f79535-47bb-0310-9956-ffa450edef68
2012-03-27 23:11:32 +00:00
Sebastian Bazley bf1d01d79d More info
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306046 13f79535-47bb-0310-9956-ffa450edef68
2012-03-27 23:10:32 +00:00
Sebastian Bazley 1337758f9f Use super-class rather than specific implementation
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1306045 13f79535-47bb-0310-9956-ffa450edef68
2012-03-27 23:10:08 +00:00
Sebastian Bazley bfc40dede6 Add CSV file parser test case runner
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1305919 13f79535-47bb-0310-9956-ffa450edef68
2012-03-27 17:32:15 +00:00
Sebastian Bazley f921e77605 Provide test access to PRISTINE
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1305705 13f79535-47bb-0310-9956-ffa450edef68
2012-03-27 01:31:33 +00:00
Sebastian Bazley aeef773ed6 Calculate the booleans once; prepares for CSV-78
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1305700 13f79535-47bb-0310-9956-ffa450edef68
2012-03-27 01:07:30 +00:00
Sebastian Bazley 8a683f009d CSV-75 ExtendedBufferReader does not handle EOL consistently
Fix tests

git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1305695 13f79535-47bb-0310-9956-ffa450edef68
2012-03-27 00:54:36 +00:00
Sebastian Bazley de1838ea06 CSV-75 ExtendedBufferReader does not handle EOL consistently
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1305694 13f79535-47bb-0310-9956-ffa450edef68
2012-03-27 00:49:17 +00:00
Sebastian Bazley 0833f45bff CSV-87 CSVParser.getRecords() returns null rather than empty List at EOF
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1305692 13f79535-47bb-0310-9956-ffa450edef68
2012-03-27 00:42:22 +00:00
Emmanuel Bourg c4e120bbf4 Added tests for line counting by the parser (CSV-75)
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1305689 13f79535-47bb-0310-9956-ffa450edef68
2012-03-27 00:20:21 +00:00
Emmanuel Bourg c6212540df Use JUnit 4 assertions for arrays
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1305674 13f79535-47bb-0310-9956-ffa450edef68
2012-03-26 23:37:47 +00:00
Sebastian Bazley 8e9fb0e989 Add more info to toString() method; use space separator for clarity
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1305669 13f79535-47bb-0310-9956-ffa450edef68
2012-03-26 23:32:18 +00:00
Emmanuel Bourg 50a89da39f Test empty files
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1305668 13f79535-47bb-0310-9956-ffa450edef68
2012-03-26 23:25:48 +00:00
Emmanuel Bourg c53d3f3025 Resistance is futile
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1305629 13f79535-47bb-0310-9956-ffa450edef68
2012-03-26 22:09:53 +00:00
Emmanuel Bourg e857ebfca1 Fixed floating semi columns
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1305548 13f79535-47bb-0310-9956-ffa450edef68
2012-03-26 20:39:13 +00:00
Sebastian Bazley 38741a48c6 CSV-54 Confusing semantic of the ignore leading/trailing spaces parameters
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1305494 13f79535-47bb-0310-9956-ffa450edef68
2012-03-26 19:02:30 +00:00
Sebastian Bazley 5063b16c7a CSV-77 RFC 4180 (DEFAULT) format is wrong; should not ignore spaces or blank lines
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1305482 13f79535-47bb-0310-9956-ffa450edef68
2012-03-26 18:31:53 +00:00
Sebastian Bazley 4406fba31c Document existing behaviour
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1304503 13f79535-47bb-0310-9956-ffa450edef68
2012-03-23 17:22:22 +00:00
Sebastian Bazley 37a261b385 Add basic toString method
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1304462 13f79535-47bb-0310-9956-ffa450edef68
2012-03-23 16:19:23 +00:00
Sebastian Bazley 43987a7d78 Rename to avoid hiding class field
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1304097 13f79535-47bb-0310-9956-ffa450edef68
2012-03-22 22:16:01 +00:00
Sebastian Bazley dd26201ac4 CSV-81 Token.Type.isReady could perhaps be removed
Not removed, but only set on EOF if there is data to return

git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1303988 13f79535-47bb-0310-9956-ffa450edef68
2012-03-22 19:04:41 +00:00
Sebastian Bazley 8bba58d1ed Don't use == null; use assertNull
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1303983 13f79535-47bb-0310-9956-ffa450edef68
2012-03-22 19:00:27 +00:00
Sebastian Bazley a1f7d8b166 Unused import
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1303982 13f79535-47bb-0310-9956-ffa450edef68
2012-03-22 18:59:52 +00:00
Sebastian Bazley 9ebd0d9425 CSV-80 - CSVLexer.nextToken does not need wsBuf
remove useless wsBuf

git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1303955 13f79535-47bb-0310-9956-ffa450edef68
2012-03-22 18:23:22 +00:00
Sebastian Bazley 1299ddfe3f Oops - fix bug introduced in r1303933
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1303948 13f79535-47bb-0310-9956-ffa450edef68
2012-03-22 18:15:20 +00:00
Sebastian Bazley fa07dea585 CSV-71 - Add convenience Methods to CSVLexer
Use convenience fields from Lexer parent class; missed one method replacement earlier

git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1303933 13f79535-47bb-0310-9956-ffa450edef68
2012-03-22 18:00:43 +00:00
Sebastian Bazley 523171ed41 Additional convenience fields
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1303929 13f79535-47bb-0310-9956-ffa450edef68
2012-03-22 17:56:03 +00:00
Sebastian Bazley 83e4a903b1 CSV-71 - Add convenience Methods to CSVLexer
Use convenience methods from Lexer parent class

git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1303904 13f79535-47bb-0310-9956-ffa450edef68
2012-03-22 17:27:51 +00:00
Sebastian Bazley ff41f78ef5 Temporary copy of original CSVLexer for performance comparison.
git-svn-id: https://svn.apache.org/repos/asf/commons/proper/csv/trunk@1303901 13f79535-47bb-0310-9956-ffa450edef68
2012-03-22 17:25:48 +00:00