Commit Graph

319 Commits

Author SHA1 Message Date
Neil Lalonde 1fdf2e4d4b badly resolved conflicts 2017-07-31 20:28:16 -04:00
Neil Lalonde fa3c240e8b Merge pull request #4981 from dmacjam/fix_limited_search_results
FIX: limited search results
2017-07-31 20:23:57 -04:00
Neil Lalonde 7c1d7fb423 Merge branch 'master' into fix_limited_search_results 2017-07-31 15:55:31 -04:00
Erick Guan f1eccd113c Replace rmmseg gem for cppjieba_rb since better dictionary (#5006)
* Rename locale to ts config in search module to make it clear

* Replace rmmese-cpp for cppjieba_rb
2017-07-31 15:28:48 -04:00
Neil Lalonde 83011045c8 fix rubocop offenses 2017-07-31 11:59:16 -04:00
Neil Lalonde 5d528f0d15 Merge pull request #4958 from dmacjam/search_posts_by_filetype
FEATURE: Search posts by filetype
2017-07-31 11:55:34 -04:00
Neil Lalonde 4a5907b116 Merge branch 'master' into search_posts_with_images 2017-07-31 10:44:41 -04:00
Guo Xiang Tan 5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Neil Lalonde ea0e90b2b0 Merge branch 'master' into search_posts_with_images 2017-07-26 10:15:55 -04:00
Guo Xiang Tan e3ac6585bd FIX: Search by topic_id should not be restricted by `SiteSetting.min_search_term_length`. 2017-07-26 09:52:39 +09:00
Neil Lalonde d8c27e3871 Merge branch 'master' into search_posts_by_filetype 2017-07-25 14:41:20 -04:00
Jakub Macina e5ee4ccc48 Add pagination and checking for more results to search. 2017-07-20 18:12:34 +02:00
Robin Ward 21e02d6969 Include the `search_log_id` in search results 2017-07-17 12:10:32 -04:00
Robin Ward 97e211f837 FEATURE: Log Search Queries 2017-07-14 14:30:58 -04:00
Jakub Macina 677267ae78 Add onceoff job for uploads migration of column extension. Simplify filetype search and related rspec tests. 2017-07-12 17:19:27 +02:00
Jakub Macina 8c445e9f17 Fix backend code for searching by a filetype as a combination of uploads and topic links. Add rspec test for extracting file extension in upload. 2017-07-06 19:19:31 +02:00
Guo Xiang Tan 7ed9fbb415 FIX: Use default parameter in method. 2017-06-26 11:14:13 +09:00
Robin Ward aaaa93c216 FIX: Full page search was removing advanced search terms 2017-06-21 15:51:15 -04:00
Jakub Macina f87d32ac6d Add backend code for searching by filetypes. 2017-06-20 21:20:06 +02:00
Jakub Macina bf002e0873 Add extraction of image_url for oneboxed images. Fix search by images filter name. 2017-06-19 15:18:02 +02:00
Neil Lalonde b98a930364 Merge pull request #4896 from dmacjam/multiple-tags-logical-and-search
FEATURE: Advanced search supports logical AND of multiple tags
2017-06-14 14:39:23 -04:00
Jakub Macina 76712da166 Add backend code for searching posts with images. 2017-06-09 10:30:21 +02:00
Guo Xiang Tan 684b05f510 FIX: Strip zero-width characters from search terms. 2017-06-07 18:19:47 +09:00
Arpit Jalan a1ebd67237 Revert "FEATURE: new setting to prioritize open topics in search" 2017-06-03 01:54:35 +05:30
Arpit Jalan b8a87a0996 FEATURE: new setting to prioritize open topics in search 2017-06-03 00:33:53 +05:30
Jakub Macina eaec35d230 Fix tag related advanced search tests into one context. Fix dynamic locale and select clause in search by all tags. Fix separator for all tags to plus sign. 2017-06-02 12:39:07 +02:00
Guo Xiang Tan 13c6695d65 Revert "Load posts in batches while indexing problem posts."
This reverts commit ce57ff9fcf.

Limit is ignored with `find_each`.
2017-06-01 11:15:27 +09:00
Jakub Macina 3eebcccbf2 Add searching by all tags using postgres full-text search. 2017-05-31 16:36:15 +02:00
Guo Xiang Tan 137f91d1cf PERF: N+1 query when searching with tags enabled. 2017-05-31 08:14:09 +09:00
Guo Xiang Tan ce57ff9fcf Load posts in batches while indexing problem posts. 2017-05-31 07:54:49 +09:00
Sam f12490eae0 FIX: order:latest not working for search within topic
FEATURE: use 'l' as a shorthand for order:latest
2017-05-24 11:24:41 -04:00
Sam 52ae63d5d7 FIX: when searching PMs also search group PMs
Users belonging to a group could not search for PMs unless explicitly added
to the PM unless admin
2017-05-11 15:59:03 -04:00
Yana Agun Siswanto cd2d2f16e5 Allow to order search results by the topic creation date
based on: https://meta.discourse.org/t/allow-to-order-search-results-by-the-topic-creation-date/38544
2017-03-30 01:18:38 +07:00
Sam 6ebddc42d1 FIX: include children categories when searching a category 2017-03-10 15:58:47 -05:00
Guo Xiang Tan c623951306 FEATURE: Search can be scoped to posts that the current user has seen/unseen.
https://meta.discourse.org/t/advanced-search-posts-that-i-have-seen/57966
2017-03-09 01:01:33 +08:00
Guo Xiang Tan 10ec554d97 Ensure we escape variables passed into our SQL query. 2017-03-08 20:37:59 +08:00
Sam Saffron df8f365d99 FEATURE: improve search so it searches sub categories by default
If you want an exact category match use `category:=howto` or `#=howto"
2017-02-07 15:53:37 -05:00
Sam 0a78ae739d Remove SearchObserver, aim is to remove all observers
rails-observers gem is mostly unmaintained and is a pain to carry forward
new implementation contains significantly less magic as a bonus
2016-12-22 13:13:14 +11:00
Sam 0631a84ca0 Merge pull request #4576 from cpradio/min-posts-search
FEATURE: Add min_post_count search filter
2016-11-29 10:19:33 +11:00
cpradio 66ca6d622e FEATURE: Add min_post_count search filter 2016-11-28 11:43:12 -05:00
Sam bc6ee85850 FIX: stop caching locale cause it bleeds in multisite 2016-11-25 11:35:29 +11:00
Sam Saffron 647ee46edf FIX: don't stem the search term
Search for "canned" not working correctly and "butted", "ands" and many more :)
2016-10-07 12:40:57 +11:00
Robin Ward b4b96bf62b FIX: Support searching your PMs via full page 2016-08-19 15:00:05 -04:00
Robin Ward ec8622a860 FIX: Count the records added, including duplicates 2016-08-12 17:18:12 -04:00
Robin Ward 11939fa8b9 PERF: Avoid some more count queries when fetching more results 2016-08-12 13:05:09 -04:00
Sam 8a792508e9 PERF: improve offset discovery query 2016-08-12 14:36:38 +10:00
Robin Ward 431c211ec2 Bump search post id cache to 1 week 2016-08-11 14:06:07 -04:00
Robin Ward 35922bf692 Use `unscoped` instead of `with_deleted` 2016-08-11 13:12:59 -04:00
Robin Ward 7e165d031b FIX: Short terms will be searched for if at least one is long enough 2016-08-11 11:53:14 -04:00
Robin Ward 4bb6f88181 FIX: Bump up backfill size to 50 2016-08-10 17:51:29 -04:00
Robin Ward c1cd9e0647 PERF: For estimates, we don't need to worry about deleted 2016-08-10 17:25:34 -04:00
Robin Ward fc311dbe3b FEATURE: An option to search more recent posts for very large sites.
On very large forums searching posts can be slow, so this commit
introduces the ability to try and search only the most recent posts
first, and then going for a larger breadth search if there aren't
enough results.

Enable `search_prefer_recent_posts` and you can customize how many
recent posts to filter with `search_recent_posts_size`
2016-08-10 15:43:42 -04:00
Robin Ward cc366d5a60 FIX: Search in non-english should have a smaller minimum 2016-08-09 15:20:28 -04:00
Robin Ward 28436a604a FIX: Prevent tricking the search from ignoring minimum lengths 2016-08-09 14:49:46 -04:00
Robin Ward c1125c8649 PERF: Use simpler serializer for search, eager load post users 2016-08-09 14:49:46 -04:00
Sam e01802a13b FIX: strip quote from search term when searching within topic 2016-07-25 15:06:25 +10:00
Robin Ward b9df18360d If you search a category by id, also include its children 2016-06-08 13:50:52 -04:00
Arpit Jalan a166869d67 FEATURE: search fallback to tags when category not found 2016-06-02 18:14:41 +05:30
Neil Lalonde f13470b96b Use db schema for tags instead of plugin store and custom fields 2016-05-26 14:29:48 -04:00
Arpit Jalan 5b030017d9 FEATURE: new search filter in:wiki 2016-05-13 13:56:53 +05:30
Arpit Jalan 00893ef1de FEATURE: new search filter - @username 2016-05-12 14:26:26 +05:30
Arpit Jalan 2e0350ee74 FEATURE: new search filter - #category-slug 2016-05-11 15:37:27 +05:30
Mittineague 66fb02acad Permit in topic search when full name is null (#4217)
* Permit in topic search when full name is null

* as per @gschlager  and  @ZogStriP pull 4217
2016-05-09 22:20:35 +02:00
Neil Lalonde e5918c7d00 FEATURE: Merge tagging plugin into core 2016-04-27 11:58:53 -04:00
Sam 77242e4680 FEATURE: in:pinned and in:unpinned search filters 2016-03-18 16:26:54 +11:00
Sam Saffron e00850a1ab FEATURE: implement before and after filters in search remove max_age and min_age
supports

- before:monday
- after:june
- before:2001
- before:2001-01-22
2016-03-14 23:27:02 +11:00
Sam Saffron 4be8de21a1 FIX: not searching group messages when searching private 2015-12-15 13:26:35 +11:00
Sam f74a6457ee FEATURE: allow CJK to be tokenized in non CJK sites.
Meaning a mixed English/Chinese site can still have a functioning search.
2015-11-27 16:35:27 +11:00
Régis Hanol 8ba5397f0d can't find staged users with search 2015-11-18 21:06:59 +01:00
Régis Hanol 7e255a151b PERF: only use fulltext when searching for a user (I checked, it's enough) 2015-11-04 23:04:37 +01:00
Régis Hanol bb79e6aff7 FEATURE: new hide_user_profiles_from_public site setting 2015-10-28 19:56:08 +01:00
Robin Ward 46bef4fad0 FIX: Allow mods/admins to search whispers 2015-09-25 11:43:04 -04:00
Sam 61ceaef4e5 FIX: allow ordering for term-less queries 2015-09-21 07:59:52 +10:00
Sam 681334701d FIX: Stop aggregating likes 2015-09-18 17:50:15 +10:00
Sam 664e2209e5 FIX: order latest was broken 2015-09-18 17:16:53 +10:00
Robin Ward 23646494eb FIX: Don't return alternative post types in search queries 2015-09-16 14:51:20 -04:00
Sam a7b34cd16b rename to badge, better more consistent name 2015-09-16 08:51:28 +10:00
Sam 6496895096 FEATURE: search for all posts by a group 2015-09-15 17:39:14 +10:00
Sam 17d999ab12 FEATURE: search for all posts of users that have a badge
with_badge:"badge name" will allow you to search only
posts of users with a particular badge
2015-09-15 17:22:08 +10:00
Sam b38a1309f7 FIX: add more quoting to avoid invalid terms 2015-08-19 09:27:47 +10:00
Sam ad2de1804e Correct bad where clause when no category/user found 2015-08-14 11:53:16 +10:00
Sam ecac786f86 FIX: remove invalid chars from ts_query helper 2015-08-13 17:55:10 +10:00
Sam 139314c213 FIX: search for 1.5 etc was broken
FEATURE: search by exact phrase

Use PG tokenizer to split up search phrase instead of relying on Ruby code
This produces much more accurate tokenization
2015-08-10 17:41:14 +10:00
Sam 41ceff8430 UX: move search to its own route
previously search was bundled with discovery, something that makes stuff confusing internally
2015-07-27 16:47:06 +10:00
Sam 4a5a2d869e FEATURE: search by user id or category id
FIX: more search results work if you open in new tab
FIX: carry context to full page search
2015-07-10 16:31:28 +10:00
Sam 9a23a0bbbd FEATURE: topic:TOPIC_ID to search within topic 2015-06-26 17:47:20 +10:00
Sam 853553f329 FIX: in:bookmarks searching deleted bookmarks 2015-06-23 13:58:23 +10:00
Sam 63351d5629 FEATURE: in:first for searching first post 2015-06-23 13:39:40 +10:00
Sam e85df6b876 FEATURE: min_age and max_age search operators 2015-06-23 13:21:50 +10:00
Sam 13f1f90c67 REFACTOR: search improved so filters are extensible
- added posts_count filter
- fixed it so you can search with a filter only
2015-06-23 12:14:06 +10:00
Sam 41e427bd2e Work in progress, full page search 2015-06-22 18:09:08 +10:00
Régis Hanol c89bef2762 FIX: proper regex for search terms 2015-05-06 19:58:29 +02:00
Sam 451788b95d FIX: allow searching in PMs when "search this topic" is selected 2015-02-19 12:56:49 +11:00
Sam bfa96811c1 remove leftover code 2015-02-05 11:54:25 +11:00
Sam 16465edd88 FEATURE: search your bookmarks using in:bookmarks 2015-02-05 11:50:20 +11:00
Sam b08f724e2b FEATURE: in:private keyword for searching private messages 2014-12-05 10:22:39 +11:00
Sam 4aa0d88c6c FEATURE: search private messages option 2014-12-04 13:50:36 +11:00
cpradio 38d901ac91 FIX: Do not show unconfirmed users in search results 2014-10-24 19:20:41 -04:00
Sam bfbfc16cca Correct hard coded URL, implement missing features 2014-10-18 18:09:45 +11:00
Sam de71477405 FEATURE: order:views order by views 2014-10-18 15:38:58 +11:00
Sam e6cc4cba8b FEATURE: in:tracking and in:watching search filters 2014-10-18 15:34:05 +11:00
Sam 8afd7a7f21 FEATURE: in:likes and in:posted search filters 2014-10-18 15:19:08 +11:00
Sam cbc132eca9 FEATURE: added status:noreplies and status:archived to advanced search 2014-10-18 14:54:11 +11:00
Sam Saffron 05eff38f34 FEATURE: add name and username to search within topic 2014-09-20 12:57:16 +10:00
Sam Saffron c5e65cc6c8 FEATURE: advanced category:NAME search operator 2014-09-19 12:44:47 +10:00
Sam 28ae3c8ad0 FEATURE: order:latest support for search 2014-09-03 22:10:18 +10:00
Sam 3a76dd3463 FEATURE: status:open and status:closed magic search strings 2014-09-03 21:54:25 +10:00
Sam f06ad7ed8e remove old unused files 2014-09-03 12:15:48 +10:00
Sam 921dd75dd9 FIX: anonymous had <a> items for pin/unpin
FEATURE: display category in search results
FEATURE: display topic state (locked/pinned/etc) in search results

UI cleanup for search results, clearing floats and so on.
2014-09-03 12:13:26 +10:00
Sam 4f09d552ed FEATURE: increase search expansion to 50 results
refactor search code to deal with proper objects
use proper serializers, test the controllers
2014-09-03 12:13:25 +10:00
Sam 69e418facf FEATURE: wider search with more context 2014-09-01 17:04:57 +10:00
Robin Ward 8a6c4234fc FIX: Re-enable searching for topic by id when using the split topic
interface.
2014-08-28 15:42:29 -04:00
Sam a1244043d3 FIX: when search finds a deep link in a topic it takes you to it 2014-08-28 17:16:39 +10:00
Robin Ward e8cade40c7 Improve search results by introducing an aggregate post search data
filter. It seems performant despite the extra content being searched.
2014-08-22 16:56:26 -04:00
Sam c1cbf1b269 PERF: new table used for title similarity search 2014-08-08 15:50:26 +10:00
Sam 16b7004767 PERF: optimise and improve topic similarity search
FIX: shows up similarity search with blank results
2014-08-08 12:12:53 +10:00
Sam aa7ed0b2ba FIX: remove broken find topic id feature 2014-07-29 16:36:52 +10:00
Sam fae3eab580 FIX: user search was stemming 2014-07-08 12:41:38 +10:00
Sam cf60bed719 FIX: username search logic was stemming and not ordering right 2014-07-05 09:11:41 +10:00
Sam 03f63d970b PERF: fix slow query 2014-07-01 22:01:15 +10:00
Sam 760e9a756d PERF: push reindex job to daily 2014-07-01 10:09:55 +10:00
Sam 90d7718103 add search filter 2014-06-30 17:18:08 +10:00
Sam 3c84876660 BUGFIX: Chinese search was broken
BUGFIX: User locale was used index data
BUGFIX: missing Norwegian fulltext config
FEATURE: store the text used to index stuff in fulltext (for diagnostics / in page search)
FEATURE: re-index posts when locale changes (in bg job)
FEATURE: allow reindexing by trucating post_search_data

Note: I removed japanese specific config cause it requires custom pg config,
  happy to add it once our base docker config ships with it
2014-06-24 17:11:13 +10:00
Sam af86014fd0 BUGFIX: bypass fulltext for search in topic 2014-06-20 15:48:34 +10:00
Sam a288ff331d BUGFIX/FEATURE: call out context for search. 2014-06-17 17:53:45 +10:00
Sam a044e3de58 Remove min_posts_for_search_in_topic
no longer needed, we always search in topic
2014-06-10 15:07:38 +10:00
Louis Rose 1574485443 Perform the where(...).first to find_by(...) refactoring.
This refactoring was automated using the command: bundle exec "ruby refactorings/where_dot_first_to_find_by/app.rb"
2014-05-06 14:41:59 +01:00
paully21 84d100be85 Add blurb of post to search results via API 2014-04-17 07:58:51 -05:00
Thomas Cioppettini 38882eb1a7 Remove threequals from ruby files 2014-03-26 12:20:41 -07:00
Robin Ward dc9003b8d4 In the event of a misconfigured locale, don't raise an exception. 2014-03-07 14:59:29 -05:00
Sam 2a8734f0d5 Suppress search in topic unless we have more than 10 posts
(configurable in site setting)
2014-02-19 08:59:18 +11:00
Sam 2b10fdc97f FEATURE: search auto scopes on topic first 2014-02-17 13:54:51 +11:00
Sam fa4a8e052f to_a not needed 2013-12-13 19:01:35 +11:00
Sam 1649f56529 work around odd scoping bug in Rails 2013-12-13 19:01:04 +11:00
German Viscuso 3f049f4853 Provides Japanese locales 2013-11-19 22:20:19 +01:00
Robin Ward 1c3c468675 FIX: Single quotes in search terms would raise an error. 2013-08-26 16:25:54 -04:00
Navin Keswani d87389b38e No more rails 4 deprecation warnings 2013-08-25 23:18:11 +02:00
Navin 2e700dd26d Patch ActiveRecord::Relation#references and clean up 2013-08-16 14:53:40 +02:00
Sam 3246f066c6 an attempt at getting this working on rails 4 ... not fruitful 2013-08-02 10:31:36 +10:00
Anton Batenev 4a6a4db241 Add russian stemmer for FTS 2013-07-23 02:05:18 +04:00
Sam ecf17cfebb work in progress, add fidelity to category group permissions (full, create posts, readonly) 2013-07-16 15:46:11 +10:00
Robin Ward 3dfb9f0c2f FIX: Searching for post content without intitial results was not working 2013-05-29 17:52:49 -04:00
Robin Ward 20e88f18ee FIX: Removes some duplicates in search results when the search context is a user. 2013-05-27 15:18:55 -04:00
Robin Ward 2dedd2bd6e When a user context is present, priortize all their posts ahead of
just the topic's OP before adding more.
2013-05-24 17:45:40 -04:00
Robin Ward 1313c0f094 FIX: When using a search context, *prefer* the context's results, don't restrict to only them. 2013-05-24 16:17:09 -04:00
Robin Ward bd779834e5 Use search context for filtering search results by current category or user 2013-05-24 14:04:26 -04:00
Robin Ward eb0c102931 Migration logic from SearchView to a controller, where it should be. 2013-05-23 17:42:57 -04:00
Robin Ward 9d0e830786 Search code now uses ActiveRecord instead of SQL. 2013-05-23 16:26:51 -04:00
Robin Ward 8e8d9af2bf Use classes instead of a complicated nested hash for search results 2013-05-23 12:44:47 -04:00
Robin Ward 7a31630837 Search Refactor: Remove some manual SQL, make search data tables more idomatic Rails/AR 2013-05-22 16:31:13 -04:00
Robin Ward b9a310f4b1 Search Refactor: Let's use a class to keep track of our state rather
than passing params everywhere. Also make the private API private.
2013-05-22 15:22:06 -04:00
Robin Ward fff46cf5aa Can search by URL or ID 2013-05-13 17:07:58 -04:00
Sam b6bf95e741 speed up startup (avoid loading some gems on startup)
correct group permission leaks
add Discourse.cache for richer caching support
2013-05-13 18:04:03 +10:00
Sam 30501e943c fix search 2013-05-13 11:07:21 +10:00
Robin Ward cf01c98d81 Experimental: Interface to Move Posts to an Existing Topic 2013-05-10 14:55:51 -04:00
Kuba Brecka 17becd0e5f update the list of i18n postgre stemmers 2013-04-10 19:45:29 +02:00
Sam 2cc6f1159c exact username match first 2013-04-02 11:58:03 +11:00
Kuba Brecka 8784c55188 implement color picking from predefined set for category badges + option to change foreground color 2013-03-14 14:16:57 +01:00
Robin Ward 88bb25f2f6 Fixes #313 - Without a slug the search urls were wrong. 2013-03-11 12:18:25 -04:00
Regis Hanol 5703d6c730 added the min-search-term-length site setting 2013-03-07 16:52:01 +01:00
Robin Ward d2596c3c4c Remove unusued site_settings, show checkbox in UI for boolean values, remove restrict_access
boolean to avoid locking yourself out by setting access_password to empty string. Minor
UI tweaks.
2013-03-01 14:27:41 -05:00
Kirill Pimenov f639397aff Correct stripping of non BasicLatin characters 2013-02-28 23:52:35 +04:00
Kirill Pimenov ee0396198e Added basic locale recognition for expected FTS stemming 2013-02-28 23:14:22 +04:00
Kirill Pimenov 2f3ba0a791 Better SQL substitutions 2013-02-28 23:03:52 +04:00
Gosha Arinich cafc75b238 remove trailing whitespaces ❤️ 2013-02-26 07:31:35 +03:00
tms eebc6c7e41 Request per_facet + 1 in search to know if we actually have more 2013-02-17 02:52:21 -05:00
Edward Ocampo-Gooding b7a76cbd24 Consistency in SQL style 2013-02-11 01:03:21 -05:00
Kuba Brecka d0c1d94918 fix user listing and search on non-US locale 2013-02-10 12:18:11 +01:00
Robin Ward 21b5628528 Initial release of Discourse 2013-02-05 14:16:51 -05:00