Commit Graph

2231 Commits

Author SHA1 Message Date
Guo Xiang Tan 71942e4f62
Merge pull request #5352 from tgxworld/method_for_replica_postgres_connection
Expose `replica_postgresql_connection` to `ActiveRecord::Base`.
2017-11-27 08:38:10 +08:00
Sam e0e99d4bbd PERF: hijack onebox requests so they do not use up a unicorn worker 2017-11-24 15:31:40 +11:00
Guo Xiang Tan a509f466a0 Expose `replica_postgresql_connection` to `ActiveRecord::Base`. 2017-11-24 09:35:45 +08:00
Gerhard Schlager 613f4d737a FIX: updating topic stats failed silently for invalid topics 2017-11-23 18:47:45 +01:00
Sam 2b8d4508e5 PERF: stop running background work between requests
Use a dedicated thread to run Scheduler::Defer

This avoids blocking of a worker during operations that require waiting.

In particular uploads risked blocking a unicorn.

This also add a queue "length" that discourse prometheus consumes.
2017-11-23 15:48:47 +11:00
Guo Xiang Tan f7642e076d REFACTOR: Avoid duplicated logic on server and client. 2017-11-21 17:10:04 +08:00
Guo Xiang Tan 385372e384 Revert "PERF: Reduce number of topics to filter while querying for unread."
This reverts commit c06b782cab.
2017-11-20 11:49:09 +08:00
Gerhard Schlager 546b206da0 replace indentation with non-breaking spaces when cooking emails 2017-11-18 17:16:44 +01:00
Neil Lalonde 87ec11e298 FIX: more accurate counting of posts read. Skipping to the end of a topic does not count all posts as read in user stats. 2017-11-17 16:08:46 -05:00
Gerhard Schlager 1a3ab7c02e ignore some site settings for emails sent to mailinglist mirror category 2017-11-17 15:29:14 +01:00
Gerhard Schlager aea161fabd suppress rejection email when email was sent to mailinglist mirror category 2017-11-17 15:29:14 +01:00
Régis Hanol 678e28794a FIX: properly handle too large & broken images in posts 2017-11-16 15:45:07 +01:00
Guo Xiang Tan 3ab6318e5f Disable postgresql adapter tests until we figure out what is leaking
connections.
2017-11-16 07:17:22 +08:00
Sam f52111f787 FEATURE: allow plugins to easily detect if running in Rack
Usage: Discourse.running_in_rack? to tell if rack was booted
2017-11-16 08:39:29 +11:00
Gerhard Schlager 9207dee69a FEATURE: escape HTML when cooking plaintext emails 2017-11-15 20:22:11 +01:00
Régis Hanol 648832a643 fix the build 2017-11-15 11:53:14 +01:00
Vinoth Kannan 7b494a65c9 NEW: large image placeholder added in cooked html (#5291) 2017-11-15 11:30:47 +01:00
Guo Xiang Tan 89d9ffa884 EXPERIMENTAL: Allow logs to be shipped via different methods.
See https://github.com/dwbutler/logstash-logger#uri-configuration for
a list of available methods.
2017-11-15 09:11:33 +08:00
Sam 813e21d0e8 FIX: current user serializer consistently returns {} for custom_fields
Resolves: #5210
2017-11-15 11:55:37 +11:00
Régis Hanol 23baaa5dcc FIX: don't crop iPhone X screenshots 2017-11-14 20:37:27 +01:00
Robin Ward 971e302ff2 FEATURE: Support an end date for user silencing 2017-11-14 13:20:19 -05:00
Gerhard Schlager d3baae5365 removes whitespaces and uses scope 2017-11-13 15:23:24 +01:00
Sam 8a66446849 FEATURE: add overflow-y auto to Markdown tables 2017-11-13 17:52:15 +11:00
Sam 4f28c71b50 FIX: error setting tombstone bucket when set to old version 2017-11-13 15:36:45 +11:00
Sam 3ac7d041ae UX: generic onebox treats all square images as avatars and renders them smaller 2017-11-13 11:21:19 +11:00
Gerhard Schlager 5210e3e744 FEATURE: accept incoming email with reply_key mismatch when original email was forwarded 2017-11-12 23:44:22 +01:00
Gerhard Schlager fc6de6863b WIP 2017-11-12 21:12:34 +01:00
Gerhard Schlager 4dc4bc70c8 FIX: ignore_by_title should match case-insensitive 2017-11-12 01:43:18 +01:00
Robin Ward 1f14350220 Rename "Blocked" to "Silenced" 2017-11-10 14:10:27 -05:00
Neil Lalonde 9dc9ca4ac0 FIX: be consistent with how first posts in topics are counted. do like DirectoryItem.refresh_period :all 2017-11-10 12:18:25 -05:00
Sam 06365023c4 FEATURE: new API to search for a user by email 2017-11-09 17:04:21 +11:00
Sam 2d5bf0705a PERF: exact email match bypass
instead of scanning full table when there is an exact email match ONLY
return the actual user.
2017-11-09 16:53:14 +11:00
Guo Xiang Tan 4bb454d889 FIX: JSON custom fields incorrectly being converted to an array.
https://meta.discourse.org/t/custom-fields-simultaneous-save-with-json-becomes-an-array/73647
2017-11-08 11:17:37 +08:00
Régis Hanol be0c7609f1 FIX: validates attachments against current authorized extensions 2017-11-07 19:17:33 +01:00
Arpit Jalan 8f560f0bf4
Merge pull request #5282 from vinothkannans/local_onebox_slash
FIX: Topic links onebox differently if end in /
2017-11-07 17:39:38 +05:30
Guo Xiang Tan d9602fe729 Fix incorrect severity field. 2017-11-07 11:24:54 +08:00
Guo Xiang Tan c9df21e131 FEATURE: Allow Unicorn logs to be JSON formatted. 2017-11-06 12:46:14 +08:00
Vinoth Kannan 230fec68ca FIX: Topic links onebox differently if end in / 2017-11-06 01:41:36 +05:30
Robin Ward 2f0c9793f1 FEATURE: Allow multiple html builders to be registered via plugins 2017-11-03 11:32:32 -04:00
ckeboss 93633865d9 Adds primary user group as a class to quote (#5285)
* Adds primary user group as a class to quote

This feature addition will add the class `group-PRIMARY_USER_GROUP` to
the quote `aside`. `PRIMARY_USER_GROUP` will be the primary user group
of the user being quoted. This is similar to the class that is added to
a `topic-post`.

* Remove trailing whitespace

* Fix avatar in test

* Address PR comments

* Fix trailing whitespace
2017-11-03 09:51:40 -04:00
Guo Xiang Tan af01860a3d Clear all active connections after PostgreSQL failover tests. 2017-11-03 09:21:10 +08:00
Neil Lalonde 21dd2ccd43 FIX: only count regular posts in user stats when deleting 2017-11-02 18:05:23 -04:00
Neil Lalonde 30689783db don't decrement post_count for a post in a deleted topic that has already been uncounted 2017-11-02 18:05:23 -04:00
Neil Lalonde f29290ad11 FIX: don't count whispers in user stats post_count 2017-11-02 18:05:23 -04:00
Neil Lalonde 24af9b7d97 FIX: when a topic is deleted, update the post count stats of all user who replied 2017-11-02 18:05:23 -04:00
Robin Ward 64cb8a3ce3 FIX: Normalizer wasn't working with attributes without values 2017-11-02 14:11:20 -04:00
Gerhard Schlager 880d154381 FIX: deleting staged user of rejected email shouldn't delete incoming email 2017-10-31 15:13:23 +01:00
Guo Xiang Tan 53cadbdfc5 Skip failing tests on Travis first. 2017-10-28 00:22:41 +08:00
Guo Xiang Tan b636e858bb Fix randomly failing spec on Travis. 2017-10-27 23:53:58 +08:00
Sam 70aed105a6 FIX: bypass all site setting work for shadowed method 2017-10-27 11:12:44 +11:00
Erick Guan 7c3123a2dd Downcase encoded slug by default and more specs 2017-10-26 16:50:29 +08:00
Guo Xiang Tan 1c7b4381cc Improve postgresql failover specs. 2017-10-26 15:54:50 +08:00
Guo Xiang Tan 94782152c6 Ensure we clean up state in PostgreSQL failover specs. 2017-10-26 09:22:51 +08:00
Sam 877b7be579 FIX: in readonly mode don't double count pages 2017-10-25 13:19:43 +11:00
Neil Lalonde fb5b9b6d06 FIX: don't offer the "Something Else" flag reason to TL0 users since they don't have permission to send private messages 2017-10-24 11:47:42 -04:00
Neil Lalonde 4452d67a23 Revert "FIX: TL0 users' messages to moderators were not being posted when flagging private messages" 2017-10-23 18:17:53 -04:00
Panayotis Matsinopoulos 821ed23799 Use the CategoryHashtag::SEPARATOR (#5258)
Use the CategoryHashtag::SEPARATOR like in the previous examples.
2017-10-23 14:59:08 +02:00
Guo Xiang Tan 54455fa40b Use MessageBus to get other processes to failover faster. 2017-10-23 17:58:24 +08:00
Guo Xiang Tan 509ecbe9fb Fix test not starting with the right state. 2017-10-23 12:20:39 +08:00
Arpit Jalan 804b4f32f8 better error message when API authentication fails 2017-10-20 20:05:34 +05:30
Guo Xiang Tan fe1e78ddf4 Make PostgreSQL failover work with distributed cache. 2017-10-20 17:15:29 +08:00
Gerhard Schlager 1481462cbf PERF: Move oneboxing from cook method "email" to postprocessing 2017-10-20 10:26:45 +02:00
Guo Xiang Tan 057571d173 Raise errors in PostgreSQLFallbackHandler threads. 2017-10-20 12:06:03 +08:00
Robin Ward 838568cbc3 Refactor flag types for more customization 2017-10-19 13:55:23 -04:00
Guo Xiang Tan 25c25ae423 FEATURE: Allow user to leave a PM. 2017-10-19 12:32:55 +08:00
Guo Xiang Tan 79de10b212 FEATURE: Allow users to disable new PMs.
https://meta.discourse.org/t/is-it-possible-to-disable-private-messaging-for-a-specific-user/46391
2017-10-19 12:32:55 +08:00
Guo Xiang Tan 814c7ab503 Skip randomly failing tests first. 2017-10-19 12:25:50 +08:00
Régis Hanol cbdfc85466 FIX: images aren't lightboxed anymore (partially reverts 646c6eb7cd) 2017-10-18 23:54:36 +02:00
Sam a4c539bade FEATURE: Allow registration of detailed request logger
Detailed request loggers can be used to gather rich timing info
from all requests (which in turn can be forwarded to monitoring solution)

Middleware::RequestTracker.detailed_request_logger(->|env, data| do
   # do stuff with env and data
end
2017-10-18 12:10:30 +11:00
Sam 1dd2b51059 remove redundent stubs 2017-10-18 12:10:30 +11:00
Guo Xiang Tan 1b5ee0ae72 FIX: Couldn't boot Discourse app with a readonly postgres. 2017-10-17 13:44:30 +08:00
Sam Saffron 8185b8cb06 FEATURE: cache https redirects per hostname
If a hostname does an https redirect we cache that so next
lookup does not incur it.

Also, only rate limit per ip once per final destination

Raise final destination protection to 1000 ip lookups an hour
2017-10-17 16:22:54 +11:00
Sam Saffron 646c6eb7cd FEATURE: add :before_post_process_cooked hook
Also reduce amount of image downloading
2017-10-17 16:22:54 +11:00
Neil Lalonde 2db66072d7 SECURITY: signup without verified email using Google auth 2017-10-16 13:51:41 -04:00
Guo Xiang Tan 5f76e5062d Pause Sidekiq when postgres failovers. 2017-10-16 19:48:31 +08:00
Neil Lalonde b124e5f19f FIX: TL0 users' messages to moderators were not being posted when flagging private messages 2017-10-13 11:55:49 -04:00
Gerhard Schlager c0bb97b5cb FIX: delete staged users when the incoming email is rejected 2017-10-11 16:17:01 +02:00
Sam 4ea87b5ab8 Merge branch 's3_refactor' 2017-10-09 10:27:52 +11:00
Régis Hanol 3bdd8f57c1 FIX: invited staged users would sometimes not get notified of replies 2017-10-06 16:37:28 +02:00
Sam 70bb2aa426 FEATURE: allow specifying s3 config via globals
This refactors handling of s3 so it can be specified via GlobalSetting

This means that in a multisite environment you can configure s3 uploads
without actual sites knowing credentials in s3

It is a critical setting for situations where assets are mirrored to s3.
2017-10-06 16:20:01 +11:00
Guo Xiang Tan b2127600fb Remove use of concurrent timer for Redis failover.
* Uses the same logic for Postgres failover.
2017-10-05 15:57:08 +08:00
Neil Lalonde f5a2ed99b0 FIX: deleting category background images sometimes has no effect 2017-10-04 17:04:37 -04:00
Neil Lalonde ddbd1d5ab8 allow regex options on username site settings 2017-10-04 15:08:51 -04:00
Neil Lalonde c29334cf23 FEATURE: the hide_email_address_taken setting works with the change email address form in user preferences 2017-10-04 11:41:25 -04:00
Sam 5881355006 remove uneeded assertion 2017-10-04 15:59:16 +11:00
Sam 4ee2fcd3d5 correct flaky spec 2017-10-04 10:47:24 +11:00
Gerhard Schlager 7f50380221 FIX: respect email domain whitelist/blacklist when creating staged users 2017-10-03 16:36:08 +02:00
Gerhard Schlager 76706f9144 FIX: don't create staged users when incoming email is rejected
FIX: don't send subscription mail to new users
2017-10-03 16:36:08 +02:00
Guo Xiang Tan 3e53dbcade UX: Only include tag hashtag postfix when necessary.
https://meta.discourse.org/t/links-to-tags-not-working-in-final-post-unless-autocompleted/69884/6?u=tgxworld
2017-10-03 13:54:50 +08:00
OsamaSayegh 79f3d299a1 Don't allow category definition topics to be converted to PMs (#5216) 2017-10-02 10:04:58 +02:00
Guo Xiang Tan b5bbb8ae8a Fix failing spec. 2017-10-02 13:16:01 +08:00
Guo Xiang Tan c872225762 Improve `MessageBus.track_publish` to allow filter by channel. 2017-10-02 11:34:57 +08:00
Sam f6fdc1ebe8 FEATURE: flexible crawler detection
You can use the crawler user agents site setting to amend what user agents
are considered crawlers based on a string match in the user agent

Also improves performance of crawler detection slightly
2017-09-29 12:31:50 +10:00
Guo Xiang Tan 8dae98a3f6 Skip randomly failing test on Travis for now. 2017-09-29 08:32:19 +08:00
Robin Ward 07e84a3afa FIX: Hack our title retriever so that it parses YouTube URLs 2017-09-28 09:30:22 -04:00
OsamaSayegh ab68379abb Use `PostDestroyer` to delete topics when performing bulk operation (#5202)
* Use `PostDestroyer` to delete topics when performing bulk operation

This PR should fix this bug: https://meta.discourse.org/t/bulk-deletion-of-topics-not-logged/50407?u=osama

* failing test
2017-09-28 14:22:49 +02:00
Sam 8ecf313a81 FIX: correctly raise errors when downloads fail
This corrects an issue where we are hitting Gravatar for 404 over and over

Also ensures file download properly reports errors
2017-09-28 16:35:43 +10:00
Guo Xiang Tan 426d2178c3 Fix undefined variable in `TopicCreator`. 2017-09-28 12:25:42 +08:00
Guo Xiang Tan d4388f54a2 FIX: Use exact patht to ensure we always redirect with the right format. 2017-09-28 10:29:41 +08:00
Guo Xiang Tan 5324c01209 FIX: Don't raise an error if reading from URL timeout. 2017-09-27 14:53:22 +08:00
Guo Xiang Tan 367fb1c524 FIX: Onebox fails on encoded URL.
https://meta.discourse.org/t/onebox-breaks-if-theres-chinese-text-in-url/67364
2017-09-26 18:34:54 +08:00
Guo Xiang Tan c25730935e FIX: Do not include non-human users on suspected users list.
https://meta.discourse.org/t/discobot-is-a-suspect-user/70558
2017-09-26 15:04:08 +08:00
Guo Xiang Tan c06b782cab PERF: Reduce number of topics to filter while querying for unread. 2017-09-26 12:36:52 +08:00
Robin Ward 561fa7d0cd FEATURE: Site Setting to hide suspension reason on the public profile 2017-09-25 12:25:14 -04:00
Robin Ward d7c37d9369 Add front end service for staff controls 2017-09-25 12:25:14 -04:00
Robin Ward 5e69217793 Add filtering support to flags 2017-09-25 12:25:14 -04:00
Guo Xiang Tan b8d9a6f747 Fix randomly failing spec. 2017-09-25 20:50:03 +08:00
Guo Xiang Tan 9540b952fd Fix broken condition in PostgresqlFallbackAdapter. 2017-09-25 13:48:59 +08:00
Guo Xiang Tan 77d4c4d8dc Fix all the errors to get our tests green on Rails 5.1. 2017-09-25 13:48:58 +08:00
Guo Xiang Tan 1a9762a9c0 Fixed private group messages being exposed in suggested topics.
Revert "Revert "PERF: Avoid unnecessary expensive joins if possible.""

This reverts commit d9714c21c8.
2017-09-15 23:40:53 +08:00
Régis Hanol 0096ee40da FIX: only show participants the user can see 2017-09-13 17:14:03 +02:00
Gerhard Schlager 31ecb4fecf FIX: Handle incoming emails without email address in From header (#5177) 2017-09-12 22:35:24 +02:00
Guo Xiang Tan 07bfe3d053 FIX: Use a distrbuted cache for site locale. 2017-09-12 19:36:17 +08:00
Guo Xiang Tan 5d4221fbe1 PERF: Avoid calling expensive `PostGuardian#can_see_post?` multiple times.
Before

```
Your Results: (note for timings- percentile is first, duration is second
in millisecs)
---
topic_admin:
  50: 19
  75: 19
  90: 21
  99: 27
topic:
  50: 56
  75: 62
  90: 64
  99: 99
timings:
  load_rails: 1262
ruby-version: 2.4.1-p111
rss_kb: 198432
pss_kb: 136612
virtual: physical
architecture: amd64
operatingsystem: Ubuntu
memorysize: 15.59 GB
kernelversion: 4.10.0
physicalprocessorcount: 1
processor0: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
rss_kb_9877: 327892
pss_kb_9877: 263671
rss_kb_9946: 325468
pss_kb_9946: 261671
rss_kb_10153: 326456
pss_kb_10153: 262657
```

After

```
Your Results: (note for timings- percentile is first, duration is second
in millisecs)
---
topic_admin:
  50: 18
  75: 18
  90: 20
  99: 28
topic:
  50: 41
  75: 42
  90: 46
  99: 49
timings:
  load_rails: 1201
ruby-version: 2.4.1-p111
rss_kb: 187936
pss_kb: 123596
virtual: physical
architecture: amd64
operatingsystem: Ubuntu
memorysize: 15.59 GB
kernelversion: 4.10.0
physicalprocessorcount: 1
processor0: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
rss_kb_26478: 342360
pss_kb_26478: 276696
rss_kb_26547: 340368
pss_kb_26547: 275930
rss_kb_26747: 338964
pss_kb_26747: 274466
```
2017-09-08 14:07:24 +08:00
Guo Xiang Tan a0daa7cad0 Oops fix build again. 2017-09-07 18:59:02 +08:00
Robin Ward db929e58fc FIX: Don't allow staff to approve users with unverified emails 2017-09-04 12:55:39 -04:00
Sam Saffron e283e6aea0 FEATURE: allowed_iframes site setting for allowing iframes
This allows you to whitelist custom iframes if needed in posts
2017-09-01 10:15:44 -04:00
Sam Saffron 3bdade8970 correct fragile spec 2017-08-31 15:55:56 -04:00
Sam c705159d22 Remove email column from user table 2017-08-29 11:50:56 -04:00
Guo Xiang Tan 37e5ee858f REFACTOR: Just pass a block instead of using a lambda. 2017-08-29 08:26:48 +08:00
Bianca Nenciu 6bc74ceb50 Split alias levels in mentionable and messageable levels. (#5065)
* Split alias levels in mentionable and messageable levels.

* Fixed some tests.

* Set messageable level to everyone by default.

* By defaults, groups are not mentionable or messageable.

* Made staff groups messageable by the system.
2017-08-28 12:32:08 -04:00
Bianca Nenciu bb3a5910d7 Support for sending PMs to email addresses (#4988)
* Added support for sending PMs to email addresses.

* Made changes after review.

* Added settings validator.

* Fixed tests.
2017-08-28 12:07:30 -04:00
Neil Lalonde 2c56f8df7c FEATURE: show tags in search results 2017-08-25 11:52:59 -04:00
Sam a4e3e081c9 correct spec 2017-08-24 10:40:14 -04:00
Guo Xiang Tan 68b8acbcfa Make Redis test less brittle. 2017-08-24 10:00:15 +09:00
Guo Xiang Tan d0f5a3d7a4 `Sidekiq::Testing.fake!` is the default mode. 2017-08-24 09:40:19 +09:00
Sam c7532003f3 add more diagnostics for flaky spec 2017-08-23 10:41:57 -04:00
Guo Xiang Tan 7c15b27a90 Merge pull request #5073 from xrav3nz/extract-github-avatar
FEATURE: import Github profile picture
2017-08-23 16:18:46 +09:00
Kyle Zhao 5868508e98 GH#retrieve_avatar: simplify conditional and restructured testing 2017-08-22 23:46:50 -04:00
Kyle Zhao 49f0119c12 FEATURE: import Github profile picture 2017-08-22 20:23:47 -04:00
Sam e8cdb1d8a7 correct flaky spec 2017-08-22 16:54:29 -04:00
Sam d4c28f6fcb remove stray puts from spec 2017-08-22 11:46:51 -04:00
Sam bcf7dc38c2 FEATURE: server side support for upload:// markdown
This allows uploads to be specified using short sha1 hash instead of full
URL

Client side change is pending
2017-08-22 11:46:23 -04:00
Guo Xiang Tan 2d4d76472d FIX: Broken suspect user lists.
https://meta.discourse.org/t/server-error-code-500/68404
2017-08-22 11:16:55 +09:00
Guo Xiang Tan e8f19e004e Use `find` instead. 2017-08-21 22:28:43 +09:00
Sam aeedecd27c correct erratically failing spec 2017-08-18 15:10:37 -04:00
Régis Hanol 629810bd07 FIX: when registering custom_field types, ensure we're casting them before checking for equality 2017-08-16 23:04:40 +02:00
Guo Xiang Tan 996ce8c443 Merge pull request #5024 from fantasticfears/site-setting
FIX: fails loud if default setting is not set
2017-08-16 09:36:38 +09:00
Erick Guan 1146772deb Fix: unlinked topic search model (#5044) 2017-08-15 11:46:57 -04:00
Erick Guan 1646bc0031 FIX: fails loud if default setting is not set
Noted:
- `push_api_secret_key` is set in initializer. Shimed with ''
- `default_theme_key` is set in seeding. Shimed with ''
2017-08-15 12:07:25 +02:00
Sam 6002f2ca4a FIX: not splitting words correctly for search tokenizer 2017-08-14 16:19:39 -04:00
Leo McArdle 0ef7a969f2 Some more HTML to Markdown fixes (#5046)
* FIX: handle spaces better within emphasis tags in html_to_markdown

* FIX: handle line breaks at beginning of emphasis tags in html_to_markdown
2017-08-14 22:13:24 +02:00
Régis Hanol 89e919ec42 UX: keep emojis in excerpts in flag queue 2017-08-14 15:16:47 +02:00
Rafael dos Santos Silva 5324c9817f FIX: Title prettify shoundn't downcase all non-ascii titles 2017-08-08 14:03:24 -03:00
Joffrey JAFFEUX 6cd8203686 FIX: allows onebox to force GET hosts returning wrong headers on HEAD 2017-08-08 11:44:27 +02:00
Guo Xiang Tan c5850422f0 FIX: SiteSettings defaults cache leaking across multisite. 2017-08-07 15:16:57 +09:00
Guo Xiang Tan 3f24ed2b3e Can't revert due to incompatibility of new site setting types.
Revert "Revert "FEATURE: Site settings defaults per locale""

This reverts commit 439fe8ba24.
2017-08-07 10:43:09 +09:00
Guo Xiang Tan 439fe8ba24 Revert "FEATURE: Site settings defaults per locale"
This reverts commit 468a8fcd20.
2017-08-07 10:31:50 +09:00
Leo McArdle a7b7fe335f FIX: allow an admin to click on blank errors (#5027)
* FIX: allow an admin to click on blank errors

* i18nlize strings

* what would a rails master do?
2017-08-04 20:04:26 +02:00
Leo McArdle 99527af38a FIX: show rejected emails with unrecognized errors (#5026)
Although 407a23663d will send rejection
messages for unrecognized errors, sometimes processing the email will
raise an error which has a blank message.

This commit:

1. Shows rejected emails which have already been processed and contain
   a blank error in /admin/email/rejected

2. Replaces new blank error messages with the error type
2017-08-04 16:20:44 +02:00
Guo Xiang Tan 8cc8010564 Maintain backwards compatibility before `Jobs::MigrateUploadExtensions` runs. 2017-08-03 11:56:55 +09:00
Guo Xiang Tan a47e297508 Merge pull request #5019 from tgxworld/more_resiliency_to_readonly_redis
Fix Redis command errors when trying to start app with a readonly Redis.
2017-08-03 06:34:36 +09:00
Sam 5942ad83c0 Skip inline onebox for domain only 2017-08-02 17:27:58 -04:00
Leo McArdle 65d5cd7239 FIX: generate valid markdown from <br></b> in an email (#5022)
* FIX: generate valid markdown from <br></b> in an email

* FIX: don't generate markdown for empty <strong> or <em> tags in emails
2017-08-02 23:02:59 +02:00
Sam a4e1920604 FIX: inline oneboxer not applying to lists 2017-08-02 16:10:08 -04:00
Sam f6bc572fb8 FEATURE: option to enable inline oneboxes for all domains
Also, change to prefer title over open graph which is often way too sparse
2017-08-02 14:27:31 -04:00
Erick Guan 468a8fcd20 FEATURE: Site settings defaults per locale
This change-set allows setting different defaults for different locales. 

It also:

- Adds extensive testing around site setting validation

- raises deprecation error if site setting has the default property based on env

- relocated site settings for dev and tests in the initializer

- deprecated client_setting in the site setting's loading process

- ensure it raises when a enum site setting being set

- default_locale is promoted to `required` category.

- fixes incorrect default setting and validation

- fixes ensure type check for site settings

- creates a benchmark for site setting

- sets reasonable defaults for Chinese
2017-08-02 12:24:19 -04:00
Guo Xiang Tan 9bc3038728 Fix Redis command errors when trying to start app with a readonly Redis. 2017-08-02 21:33:56 +09:00
Sam 97fa64d846 FIX: non tag/category # searches should be passed through 2017-08-01 18:15:14 -04:00
Sam 71ad3a48c2 Correct flaky distributed cache test
make distributed cache more testable
2017-08-01 16:11:48 -04:00
Guo Xiang Tan 5b76b9e888 Ensure that we cleanup after tests. 2017-08-01 23:42:26 +09:00
Guo Xiang Tan 7d5b6e2b3c Disable MessageBus's keepalive when Redis is readonly. 2017-08-01 23:07:52 +09:00
Neil Lalonde 24edbbb89c fix rubocop offenses 2017-07-31 20:34:02 -04:00
Neil Lalonde fa3c240e8b Merge pull request #4981 from dmacjam/fix_limited_search_results
FIX: limited search results
2017-07-31 20:23:57 -04:00
Régis Hanol 88ba052446 secure default for the 'find_related_post_with_key' site setting 2017-08-01 00:03:04 +02:00
Sam cdf7a020c1 correct mistake in spec 2017-07-31 16:40:59 -04:00
Neil Lalonde 7c1d7fb423 Merge branch 'master' into fix_limited_search_results 2017-07-31 15:55:31 -04:00
Erick Guan f1eccd113c Replace rmmseg gem for cppjieba_rb since better dictionary (#5006)
* Rename locale to ts config in search module to make it clear

* Replace rmmese-cpp for cppjieba_rb
2017-07-31 15:28:48 -04:00
Neil Lalonde 5d528f0d15 Merge pull request #4958 from dmacjam/search_posts_by_filetype
FEATURE: Search posts by filetype
2017-07-31 11:55:34 -04:00
Neil Lalonde 4a5907b116 Merge branch 'master' into search_posts_with_images 2017-07-31 10:44:41 -04:00
Jakub Macina f8edf2636c Fix rspec tests for search pagination. 2017-07-31 12:16:23 +02:00
Régis Hanol c7f76aa554 fix the receiver_spec 2017-07-31 11:29:39 +02:00
Arpit Jalan b059a0f789 extract url escaping to a dedicated class method and improved tests 2017-07-29 22:16:51 +05:30
Arpit Jalan 1fe553873c FIX: preserve fragment identifier when escaping url 2017-07-29 17:22:45 +05:30
Sam 56fd3e1cd4 attempt to stablise flaky spec 2017-07-28 16:59:37 -04:00
Robin Ward 9a76559345 FIX: Remove unused test 2017-07-28 11:59:12 -04:00
Guo Xiang Tan 5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Sam 4f574e7c93 FEATURE: support inline bbcode blocks eg: [quote]test[/quote] 2017-07-27 17:55:17 -04:00
Neil Lalonde 68b3dd43ce fix intermittent failing tests, some watched word refactoring 2017-07-27 12:27:01 -04:00
Sam dc42d4411d correct flaky spec 2017-07-26 13:22:19 -04:00
Neil Lalonde 24cb950432 FEATURE: Watched Words: when posts contain words, do one of flag, require approval, censor, or block 2017-07-26 11:01:09 -04:00
David Taylor 27539e15e1 Allow whitespace around % scaler in markdown image syntax (#5000) 2017-07-26 10:54:20 -04:00
Neil Lalonde ea0e90b2b0 Merge branch 'master' into search_posts_with_images 2017-07-26 10:15:55 -04:00
Guo Xiang Tan d3a975e99a Improve specs to check top level stylesheets for SCSS compilation errors.
https://meta.discourse.org/t/fwd-growth-institute-pm-error-in-css-on-embed/66824/4
2017-07-26 10:50:51 +09:00
Guo Xiang Tan 96267f0845 Merge pull request #4960 from discourse/category-description-error
FIX: Explicit error when category description post is bad
2017-07-26 08:54:22 +09:00
Sam ad8643bfe2 attempt to stablise spec in travis 2017-07-25 16:58:19 -04:00
Sam ccbcbd0343 remove 2 failures that happen if discobot booted 2017-07-25 16:08:03 -04:00
Neil Lalonde d8c27e3871 Merge branch 'master' into search_posts_by_filetype 2017-07-25 14:41:20 -04:00
Sam f2d5bcd439 Stop writing to STDOUT/ERR during tests 2017-07-25 11:37:25 -04:00
Guo Xiang Tan 4cd9822611 Remove warnings in `ColumnDropper.mark_readonly` test. 2017-07-25 09:38:55 +09:00
Guo Xiang Tan be3c9bca0b Remove comment that is no longer relevant. 2017-07-25 08:58:40 +09:00
Sam 4da98cdcfe FIX: allow bbcode to be unquoted for default 2017-07-24 18:36:17 -04:00
Sam 04eac9f14a lets attempt to get these specs working! 2017-07-24 18:35:20 -04:00
Sam f67e715ef1 comment out specs that break others
will check in a fixed spec tomorrow
2017-07-24 17:28:24 -04:00
Sam c08a7aee8f clean up skipped tests
tighter connection handling in scheduler
2017-07-24 15:06:24 -04:00
Sam 80bb127cd3 use time zone for extra safety 2017-07-24 13:08:17 -04:00
Sam 7ebb9874a8 freeze time in spec 2017-07-24 13:00:15 -04:00
Sam 40abcfc2f5 FIX: smart quote edge case with quotes 2017-07-24 12:21:49 -04:00
Sam Saffron 045a2abcec FEATURE: remove the timecop gem
We should only have one way of mocking time, misuse of timecop
was causing build stability issues
2017-07-24 12:11:10 -04:00
Guo Xiang Tan d940166a89 Re-enable skipped `Scheduler::ScheduleInfo` test. 2017-07-25 00:03:03 +09:00
Guo Xiang Tan 8ee31d3c85 Re-enable skip tests for `Email::Styles`. 2017-07-24 23:16:54 +09:00
Guo Xiang Tan e82efc7b03 Fix `SiteSetingExtension` specs to coerce int to string. 2017-07-24 22:54:42 +09:00
Guo Xiang Tan 0ab0c82f11 Reenable skipped `Email::Receiver` spec. 2017-07-24 22:27:24 +09:00
Guo Xiang Tan d9c3ebe299 Reintroduce specs that wasn't causing random failures. 2017-07-24 19:32:38 +09:00
Régis Hanol 640ced004d FIX: wasn't unfreezing time in email processor spec 2017-07-24 11:01:53 +02:00
Guo Xiang Tan 1b0750d7ef Merge pull request #4983 from tgxworld/group_owners_can_invite_users_to_groups
Group owners can invite users to groups
2017-07-24 16:21:19 +09:00
Guo Xiang Tan fa09a02201 Fix specs. 2017-07-24 15:56:08 +09:00
Sam Saffron b9757bd02e remove readonly column specs for now 2017-07-22 22:20:57 -04:00
Robin Ward 2f8f2aa1dd FEATURE: Whitelists for inline oneboxing 2017-07-21 15:41:47 -04:00
Rafael dos Santos Silva 89ef5d36a9 FIX: Explicit error when category description post is bad 2017-07-21 16:07:29 -03:00
Régis Hanol fff5e2c3a5 Merge pull request #4984 from LeoMcA/unrecognized-error-email
FEATURE: send rejection email for unrecognized errors
2017-07-21 20:40:18 +02:00
Robin Ward 574681dc47 FIX: Show emoji in inline oneboxes 2017-07-21 14:24:48 -04:00
Leo McArdle 407a23663d FEATURE: send rejection email for unrecognized errors 2017-07-21 18:26:52 +01:00
Sam e1ce47a901 Pass the full CommonMark spec 2017-07-21 13:20:52 -04:00
Guo Xiang Tan 2a17f1ccd7 FIX: Group owners should be able to invite users to their groups.
https://meta.discourse.org/t/group-owner-cannot-send-an-invite-to-a-group/60617/12
2017-07-21 23:48:25 +09:00
Jakub Macina 4eb7f7cd10 Add rspec tests for search pagination. 2017-07-21 10:43:02 +02:00
Sam fc90dd3592 Work in progress, CommonMark spec 2017-07-20 17:47:16 -04:00
Robin Ward d2490cbbb8 Test failures for Inline Onebox 2017-07-20 16:01:16 -04:00
Robin Ward 3882722195 FEATURE: Inline (Mini) Oneboxing
see:
https://meta.discourse.org/t/mini-inline-onebox-support-rfc/66400?source_topic_id=66066
2017-07-20 15:38:04 -04:00
Sam 44fb2a2833 DEV: support multiple capture groups for text post process 2017-07-20 15:33:44 -04:00
Sam e7c170bb00 test v8 heap stats and aggregate all 2017-07-20 13:23:48 -04:00
Sam 1096dcd602 correct bbcode parsing edge case 2017-07-20 13:02:40 -04:00
Sam 911ffbb98c Correct mention edge cases
@sam. was not correctly detected as a mention
2017-07-20 11:52:38 -04:00
Leo McArdle d0b027d88d FEATURE: phase 1 of supporting multiple email addresses 2017-07-20 11:22:27 +09:00
Guo Xiang Tan b534778f46 FIX: Escape URL before attempting to resolve it. 2017-07-18 10:04:24 +09:00
Sam 627f5ddbd1 Correct specs 2017-07-17 18:00:25 -04:00
Sam c7b0764089 correct more edge cases with new engine 2017-07-17 17:44:48 -04:00
Sam 8921058c67 FIX: medium URL with @ incorrectly handled as mention
Also:

- remove unused code
- rename bbcode_ruler to bbcode.ruler
- add md.core.textPostProcess.ruler to apply at end of chain (excluding links)
2017-07-17 17:44:48 -04:00
Robin Ward 21e02d6969 Include the `search_log_id` in search results 2017-07-17 12:10:32 -04:00
Sam Saffron d0c5205a52 Feature: Change markdown engine to markdown it
This commit removes the old evilstreak markdownjs engine.

- Adds specs to WhiteLister and changes it to stop using globals
    (Fixes large memory leak)
- Fixes edge cases around bbcode handling
- Removes mdtest which is no longer valid (to be replaced with
    CommonMark)
- Updates MiniRacer to correct minor unmanaged memory leak
- Fixes plugin specs
2017-07-17 11:41:34 -04:00
Sam 79a084dd58 Revert "remove old markdown engine work-in-progress"
This reverts commit ee470b5317.
2017-07-12 18:10:51 -04:00
Sam bcbb9f208d Revert "Integrate new engine, correct old specs"
This reverts commit f1b38ba4fb.
2017-07-12 18:10:07 -04:00
Sam f1b38ba4fb Integrate new engine, correct old specs
corrects edge cases with

- full quotes
- [url] with nested tags
- engine overrides
- onebox applying to non http srcs
2017-07-12 17:44:40 -04:00
Sam Saffron ee470b5317 remove old markdown engine work-in-progress 2017-07-12 17:44:40 -04:00
Jakub Macina 677267ae78 Add onceoff job for uploads migration of column extension. Simplify filetype search and related rspec tests. 2017-07-12 17:19:27 +02:00
Sam 5d139e461c FIX: multi option poll not working (new engine) 2017-07-11 14:43:34 -04:00
Bianca Nenciu 1c917cc391 Added test. 2017-07-11 13:59:37 -04:00
Sam 3f950a756a FEATURE: support image dimensions via Markdown image 2017-07-11 12:13:03 -04:00
Guo Xiang Tan 45f4ce379e Fix broken specs. 2017-07-11 13:16:48 +09:00
Guo Xiang Tan b605d5d61b FIX: Translation should return overrides first before attempting to fallback.
https://meta.discourse.org/t/errors-after-migrating-to-a-new-version-1-9-0-beta3/65709/14?u=tgxworld
2017-07-11 12:51:12 +09:00
Sam ba9898c5a9 FIX: smarter newline handling for <img> tags on line alone
Run all of pretty text spec on new engine
2017-07-10 12:22:15 -04:00
Guo Xiang Tan a338a7a53e Fix the build. 2017-07-10 11:12:21 +09:00
Guo Xiang Tan 2255724637 UX: Add validator for `SiteSetting#sso_overrides_email`. 2017-07-10 10:08:55 +09:00
Blake Erickson fcfc895167 FIX: new sign-ups via google are added to groups
This fix ensures that users that are signing up via google oauth are
automatically added to any groups.

A similar fix will probably need to be added to other oauth providers.
2017-07-07 14:08:58 -06:00
Sam 6f09df0deb FIX: don't create tags autolinks inside links (new engine) 2017-07-07 13:04:25 -04:00
Sam 74f6894606 oops checked in a broken test 2017-07-07 11:08:04 -04:00
Sam 436b894f7a FIX: mention not working after a newline (new engine) 2017-07-07 11:06:50 -04:00
Guo Xiang Tan 13f3de4bf6 Nuke all `SiteSetting.stubs` from our codebase. 2017-07-07 15:09:14 +09:00
Guo Xiang Tan a4399c2eab FIX: Translation overrides were not cached by locale properly. 2017-07-07 12:28:00 +09:00
Arpit Jalan 7cffbc8ba8 FEATURE: new site setting to limit message recipients
New site setting `max_allowed_message_recipients` to limit message
recipients

https://meta.discourse.org/t/one-of-my-users-just-group-messaged-100-other-user-with-a-spam-offer/65612/7?u=techapj
2017-07-06 22:52:49 +05:30
Jakub Macina f0a674d620 Add extraction of upload extension. Add rspec test for search of post
with upload by extension.
2017-07-06 19:16:07 +02:00
Guo Xiang Tan a7a8ccb100 Merge pull request #4948 from tgxworld/fix_translation_fallback_not_using_overrides
FIX: Translation fallback was not using fallback's translation override.
2017-07-04 09:08:22 +09:00
Sam d2ba543ed3 correct case where %20 decoded as space in auto link
correct case where onebox is not applied when url is decoded
2017-07-03 16:32:53 -04:00
Sam 845170bd6b FEATURE: add support for group visibility level
There are 4 visibility levels

- public (default)
- members only
- staff
- owners

Note, admins and group owners ALWAYS have visibility to groups

Migration treated old "non public" as "members only"
2017-07-03 15:26:57 -04:00
Sam c79418d334 DEV: Move RTL into lib where it belongs 2017-07-03 15:26:57 -04:00
Guo Xiang Tan 52e654b3ac FIX: Translation fallback was not using fallback's translation override.
https://meta.discourse.org/t/discobot-falling-back-to-site-locale-is-not-using-customized-copies/65140
2017-07-03 14:52:27 +09:00
Sam d941ed90d6 support for bbcode [code] blocks 2017-06-29 16:04:10 -04:00
Sam de50d8cd35 Support for [email] [url] [img] bbcodes 2017-06-29 14:00:09 -04:00
Sam c6b83f4714 Correct logic so hashtags are properly ignored in links 2017-06-29 12:48:20 -04:00
David Taylor 9294826f31 Allow inclusion of elided text from emails when sending to a category… (#4844)
* Allow inclusion of elided text from emails when sending to a category address
2017-06-29 13:03:14 +09:00
Sam 0e391a72a1 use custom whitelister for tables 2017-06-28 16:08:29 -04:00
Sam cfbda863e8 implement unicode emoji replacements 2017-06-28 13:47:30 -04:00
Sam 4c5109ff5b FEATURE: site setting for Markdown typographer
It ships anyway with markdown.it so we might as well expose it
2017-06-27 16:50:13 -04:00
Sam 0d62420cbe Space prior to onebox stops onebox resolution 2017-06-27 16:16:17 -04:00
Sam 232de90607 explicit autolink should not be oneboxed 2017-06-27 15:34:16 -04:00
Sam 5897ae945a FIX: stop using inline quote for local onebox 2017-06-27 10:57:38 -04:00
Sam 0e0092d738 corrected a parsing edge cases for bbcode blocks 2017-06-27 10:07:06 -04:00
Régis Hanol cdf8baac3b fix specs due to new default emoji set 2017-06-26 23:36:30 +02:00
Sam 21f7ac1f25 correct spec, require closing tag for bbcode blocks 2017-06-26 16:51:55 -04:00
Robin Ward db485ae0da FIX: Support for skipping redirects on certain domains (like steam) 2017-06-26 15:38:43 -04:00
Sam 8967d50dc2 Clean up sanitization code
- remove html table test, this is soon to be deprecated
- move sanitization tests into pretty text.rb
- fix up whitelister so it makes a copy of options
2017-06-26 15:21:47 -04:00
Neil Lalonde 69b2d64334 FIX: letter avatars in quotes on subfolders installs are broken 2017-06-23 17:12:13 -04:00
Sam 814c8804d4 correct text highlighting for no highlight 2017-06-23 14:35:46 -04:00
Sam b7386958ed don't leak state into other tests 2017-06-23 12:01:33 -04:00
Sam 234694b50f Feature: CommonMark support
This adds the markdown.it engine to Discourse.
https://github.com/markdown-it/markdown-it

As the migration is going to take a while the new engine is default
disabled. To enable it you must change the hidden site setting:
enable_experimental_markdown_it.

This commit is a squash of many other commits, it also includes some
improvements to autospec (ability to run plugins), and a dev dependency
on the og gem for html normalization.
2017-06-23 12:01:33 -04:00
Neil Lalonde 8fd915a11a Revert "FIX: add path to cookies so sessions on the same domain but different subfolders don't log each other out" 2017-06-21 16:18:24 -04:00
Robin Ward aaaa93c216 FIX: Full page search was removing advanced search terms 2017-06-21 15:51:15 -04:00
Arpit Jalan 6796de8bf2 FIX: replace site_name with email_prefix in subject when use_site_subject is true 2017-06-21 21:15:49 +05:30
Jakub Macina aad1c9bef2 Add rspec tests for searching by a filetype. 2017-06-20 21:21:56 +02:00
Neil Lalonde 1716747810 FIX: add path to cookies so sessions on the same domain but different subfolders don't log each other out 2017-06-20 13:30:36 -04:00
Jakub Macina bf002e0873 Add extraction of image_url for oneboxed images. Fix search by images filter name. 2017-06-19 15:18:02 +02:00
Alberto Aguirre b24bdc5b20 FIX: discourse_hub_spec stub_request should include query params 2017-06-18 14:40:28 -05:00
Neil Lalonde c5f42f0e97 fix discourse_hub_spec 2017-06-16 15:31:13 -04:00
Guo Xiang Tan a6450f7377 REFACTOR: Allow `delete_removed_posts_after` SiteSetting to be bypassed. 2017-06-16 12:27:51 +09:00
Guo Xiang Tan d82dbd565b FIX: Remove dependency on `rest-client`. 2017-06-16 09:42:41 +09:00
Neil Lalonde eb9f306e64 FIX: fail to send email notification if post has img tag with no src attribute 2017-06-15 10:36:19 -04:00
Sam ac1f84d3e1 SECURITY: theme key should be an anon cache breaker 2017-06-15 09:36:27 -04:00
Joffrey JAFFEUX f1750be2be Fixes woman_shrugging name conversion 2017-06-15 08:26:29 +02:00
Neil Lalonde b98a930364 Merge pull request #4896 from dmacjam/multiple-tags-logical-and-search
FEATURE: Advanced search supports logical AND of multiple tags
2017-06-14 14:39:23 -04:00
Joffrey JAFFEUX 95e1843d33 Improves native emoji conversion 2017-06-14 17:42:52 +02:00
Joffrey JAFFEUX ab6086bce9 Transforms native skin toned emojis to valid names 2017-06-14 15:35:37 +02:00
Robin Ward 009f0921dc FEATURE: Whitelist hosts for internal crawling 2017-06-13 12:59:54 -04:00
Sam 826f332766 Merge pull request #4901 from jjaffeux/emoji-5.0
Emoji next
2017-06-13 11:12:41 -04:00
Robin Ward a3729b51eb FIX: Always allow the host the forum is hosted on 2017-06-12 13:22:51 -04:00
Guo Xiang Tan eab3562864 Merge pull request #4905 from tgxworld/strip-zero-width-chars-from-search-term
FIX: Strip zero-width characters from search terms.
2017-06-12 18:01:02 +09:00
Guo Xiang Tan e888369f51 UX: Don't send emails for discobot notifications. 2017-06-12 17:00:27 +09:00
Robin Ward b5bead1fe3 FIX: Don't use `target=_blank` for local oneboxes 2017-06-09 12:15:36 -04:00
Jakub Macina 76712da166 Add backend code for searching posts with images. 2017-06-09 10:30:21 +02:00
Robin Ward 602e4d0da4 FIX: Return `nil` when the `reply_key` is missing 2017-06-08 14:28:48 -04:00
Guo Xiang Tan 684b05f510 FIX: Strip zero-width characters from search terms. 2017-06-07 18:19:47 +09:00
Robin Ward 53b95f009f FIX: If HEAD is not supported, try GET. Also set cookies 2017-06-06 13:53:49 -04:00
Jakub Macina 8f7d81fde6 Add rspec test for searching posts with images. 2017-06-06 14:39:53 +02:00
Joffrey JAFFEUX 09d34b7642 emoji images are now v5 not v3
TODO: we should not hardcode this
2017-06-05 16:10:36 +02:00
Arpit Jalan a1ebd67237 Revert "FEATURE: new setting to prioritize open topics in search" 2017-06-03 01:54:35 +05:30
Arpit Jalan b8a87a0996 FEATURE: new setting to prioritize open topics in search 2017-06-03 00:33:53 +05:30
Jakub Macina eaec35d230 Fix tag related advanced search tests into one context. Fix dynamic locale and select clause in search by all tags. Fix separator for all tags to plus sign. 2017-06-02 12:39:07 +02:00
Guo Xiang Tan bd486100c0 Remove stubs on DiscourseEvent in tests. 2017-06-01 16:21:00 +09:00
Jakub Macina 6d1fafaff6 Add rspec tests for advanced search by tags. 2017-05-30 13:01:17 +02:00
Sam 0aed2533ac Revert unread optimisation, has too many edge cases 2017-05-26 09:04:13 -04:00
Guo Xiang Tan 56f98de7b2 Use webmock to stub external web requests. 2017-05-26 15:19:09 +08:00
Guo Xiang Tan f8f1548fd4 Revert "FIX: Use Excon to do its own stubbing"
This reverts commit 80af54460a.
2017-05-26 13:04:25 +08:00
Sam 29fac1ac18 PERF: improve performance of unread queries
Figuring out what unread topics a user has is a very expensive
operation over time.

Users can easily accumulate 10s of thousands of tracking state rows
(1 for every topic they ever visit)

When figuring out what a user has that is unread we need to join
the tracking state records to the topic table. This can very quickly
lead to cases where you need to scan through the entire topic table.

This commit optimises it so we always keep track of the "first" date
a user has unread topics. Then we can easily filter out all earlier
topics from the join.

We use pg functions, instead of nested queries here to assist the
planner.
2017-05-25 15:07:30 -04:00
Robin Ward cdbe027c1c Refactor `FileHelper` to use keyword arguments. 2017-05-24 13:54:26 -04:00
Sam f12490eae0 FIX: order:latest not working for search within topic
FEATURE: use 'l' as a shorthand for order:latest
2017-05-24 11:24:41 -04:00
Robin Ward 3b0cbf7013 FIX: Always allow downloads from CDN 2017-05-23 16:32:54 -04:00
Robin Ward b81e7be9a1 FEATURE: Rate limit how often we'll crawl a destination IP 2017-05-23 15:03:04 -04:00
Robin Ward 36e477750c FIX: Use same code path for downloading images 2017-05-23 14:51:30 -04:00
Robin Ward e5e7a15a85 SECURITY: Never crawl by IP 2017-05-23 13:07:18 -04:00
Robin Ward 93a5fc62bf FEATURE: A site setting to prevent crawling on private IP blocks 2017-05-23 11:56:06 -04:00
Robin Ward 80af54460a FIX: Use Excon to do its own stubbing 2017-05-22 18:19:20 -04:00
Robin Ward b51126dd5e FIX: Reset the WebMock after before every test 2017-05-22 17:52:31 -04:00
Régis Hanol 9dddb81cf6 FIX: remove memoization on class method used in a job 2017-05-22 23:35:41 +02:00
Blake Erickson 4e8beda332 Merge pull request #4866 from JaredReisinger/admin-user-pagination
Add pagination to /admin/users/list API
2017-05-22 15:12:59 -06:00
Robin Ward d4b16b487e FIX: Another onebox head request 2017-05-22 17:00:19 -04:00
Robin Ward a8d1e44943 FIX: Onebox will do a HEAD request first for redirects 2017-05-22 16:52:26 -04:00
Robin Ward 4c690f7089 Use `FinalDestination` to ensure public redirects for onebox 2017-05-22 16:42:49 -04:00
Robin Ward b23fc2bf84 Helper to find the final destination for a URL 2017-05-22 15:52:41 -04:00
Régis Hanol a7f337fa93 FIX: CDN wasn't properly applied to category background images when using S3 2017-05-22 18:37:01 +02:00
Robin Ward 28f486cb7a FIX: Regular users shouldn't be able to invite to PMs if disabled 2017-05-19 12:57:21 -04:00
Régis Hanol 4fb335f1f0 FIX: don't process the same incoming email more than once 2017-05-18 16:43:07 +02:00
Régis Hanol a1b8a3b52b FIX: supports bare <li> when converting html to markdown 2017-05-17 15:05:11 +02:00
Jared Reisinger 645f648ea7 Merge remote-tracking branch 'upstream/master' into admin-user-pagination 2017-05-16 13:36:47 -07:00
Neil Lalonde 7821400141 FEATURE: staff can set a timer to remind them about a topic 2017-05-16 14:49:50 -04:00
Robin Ward dd0cb6a117 FIX: Add missing test 2017-05-15 16:57:09 -04:00
Jared Reisinger 1dcd61fa34 Add pagination to /admin/users/list API
Prior to this, only the first 100 active/new/etc. users were available
via the `/admin/users/list` API.  This change adds support for a
`page=#` querystring parameter so that *all* of the users can be
retrieved.  Requests for pages past the last user result in an
empty-list response; requests for negative pages (or zero) just return
the first page.

Added tests to cover pagination.
2017-05-12 13:28:35 -07:00
Robin Ward 1fb08d24d9 FIX: Latest onebox adds more attributes to links for security 2017-05-12 15:49:56 -04:00
Neil Lalonde 8b9799507c fix deploy problems due to renamed table 2017-05-12 15:17:25 -04:00
Sam 47ce674798 PERF: bypass wizard check after 15 topics are created 2017-05-12 10:18:43 -04:00
Neil Lalonde 55b61e9bea rename topic_status_update to topic_timer 2017-05-11 18:27:53 -04:00
Sam 52ae63d5d7 FIX: when searching PMs also search group PMs
Users belonging to a group could not search for PMs unless explicitly added
to the PM unless admin
2017-05-11 15:59:03 -04:00
Sam 9b84cd832c confirm in:private works for non staff 2017-05-11 15:06:45 -04:00
Robin Ward b57b635d30 FIX: Extract `div` tags within `span`s 2017-05-09 12:33:54 -04:00
Robin Ward addc85cd08 FIX: Don't let users edit wiki posts unless they can reply 2017-05-08 16:23:11 -04:00
Robin Ward 009e120e13 FIX: Don't raise an error when the img attribute cannot be found 2017-05-08 15:35:31 -04:00
Robin Ward 96c59c5b82 FIX: Don't notify or return flags on system users 2017-05-08 13:13:35 -04:00
Sam 7a15b44ac3 rename on_remove, on_drop 2017-05-04 14:03:17 -04:00
Sam 6a6eed4ed2 DEV: column dropper class for cleaner removal of superflous columns
Also fixes issues during deploy cause target column was renamed in
theme_fields
2017-05-04 10:15:41 -04:00
Guo Xiang Tan 4101db8b36 FIX: Invalid creation of `Theme` in wizard builder. 2017-05-04 11:44:23 +08:00
Régis Hanol bf322281e3 Improve handling of inlined images in incoming emails 2017-05-03 23:02:02 +02:00
Régis Hanol 768c63c103 Add 'keep_cid_imgs' option to HTML to Markdown converter to improve incoming email parsing 2017-05-03 23:01:55 +02:00
Régis Hanol e38014772b FIX: skip hidden <img> (no tracking for you) 2017-05-03 19:40:34 +02:00
Régis Hanol c8044c6956 FIX: skip hidden nodes when converting from HTML to Markdown 2017-05-03 19:34:03 +02:00
Régis Hanol bff36de130 FIX: HtmlToMarkdown should not convert empty/bad <img> tags 2017-05-03 18:29:25 +02:00
Régis Hanol c880af8120 FIX: properly trim whitespaces (including those pesky &nbsp; html entities) 2017-05-03 18:04:31 +02:00
Sam 342ef5f81a FEATURE: out-of-the-box dark/light user selectable themes 2017-05-03 11:31:33 -04:00
Régis Hanol edbf12622b FIX: HtmlToMarkdown should not convert empty/bad <a> tags 2017-05-03 16:42:37 +02:00
Sam 946f25098f Refactor theme fields so they support custom theme defined vars
This paves the way to allowing themes to specify uploads and so on.
2017-05-02 16:02:14 -04:00
Sam ba9339b8d7 FIX: missing image-url helper 2017-05-01 15:32:02 -04:00
Robin Ward b99aedeccc Merge pull request #4838 from vinothkannans/fix/read
FIX: 'read' filter in top menu showing new topics
2017-05-01 11:21:13 -04:00
Claus Strasburger e9bb9a167b Fix reply-by-mail for 8-bit transfer encodings
The mail class seems to handle mails sent with Content-Transfer-Encoding: 8bit
somewhat weirdly: It decodes them (to utf-8), changes the raw source to base64,
and does not modify the Content-Type:charset= header.

This leads to Discourse trying the message encoding (in my example ISO-8859-1)
first, and if that does not contain any unparseable characters, it uses that.
Sadly, in ISO-8859-1, every byte sequence is valid.

Fix this by always trying to decode as UTF-8 first. The probability of someone
using another encoding that cleanly (but wrongly) decodes as UTF-8 should be
fairly low.
2017-04-30 23:30:40 +02:00
Vinoth Kannan 1241660c2c FIX: 'read' filter in top menu showing new topics 2017-04-29 14:37:45 +05:30
Régis Hanol aba76bace6 add support to keep img tags when converting to html 2017-04-28 22:14:46 +02:00
Régis Hanol 51ee49aad2 FIX: properly support HTML document when converting to markdown 2017-04-28 22:02:20 +02:00
Régis Hanol 0ec15af970 restore the 'incoming_email_prefer_html' site setting 2017-04-27 14:31:11 +02:00
Guo Xiang Tan 59b906ab0d FEATURE: Disable minimum post length check when in PM with non human users.
https://meta.discourse.org/t/discourse-narrative-bot-beta-feedback/58621/65?u=tgxworld
2017-04-27 16:00:22 +08:00
Régis Hanol b76674f640 FEATURE: convert incoming emails in HTML to markdown
- remove incoming_email_prefer_html site setting
- remove HtmlCleaner class
2017-04-26 16:49:06 +02:00
Guo Xiang Tan aef89c4850 REFACTOR: Load `Post` records in batches when destroying stubs. 2017-04-25 10:19:21 +08:00
Régis Hanol d5630d6160 HtmlToMarkdown library
Small library to transform HTML to Discourse-flavored markdown (mostly used for imports)
2017-04-24 22:01:41 +02:00
Arpit Jalan dad2024094 FIX: do not impose default min/max validation on hidden site setting 2017-04-22 12:08:39 +05:30
Sam b077335a30 make stat socket much more robust 2017-04-21 17:12:29 -04:00
Arpit Jalan b0151ab66a Merge pull request #4826 from techAPJ/site-setting-max-value
FIX: all basic integer settings should have min & max value validation
2017-04-21 20:21:10 +05:30
Sam 52306c393a FEATURE: basic implementation of stats socket 2017-04-21 10:24:43 -04:00
cpradio 20c2c66dd4 FEATURE: Add normal as a preference for topic subscription state when replying to a topic 2017-04-20 22:33:10 -04:00
Arpit Jalan 9eff4f0807 FIX: all basic integer settings should have max value validation 2017-04-21 07:09:41 +05:30
Sam e119c6e01e FIX: embedded comments not working when theme is missing 2017-04-20 10:31:33 -04:00
Robin Ward 1363988cd7 Support for an HTML builder that can create dynamic HTML 2017-04-17 17:32:55 -04:00
Guo Xiang Tan 04016f0dec Support Ruby 2.4. 2017-04-15 12:29:00 +08:00
Sam a3e8c3cd7b FEATURE: Native theme support
This feature introduces the concept of themes. Themes are an evolution
of site customizations.

Themes introduce two very big conceptual changes:

- A theme may include other "child themes", children can include grand
children and so on.

- A theme may specify a color scheme

The change does away with the idea of "enabled" color schemes.

It also adds a bunch of big niceties like

- You can source a theme from a git repo

- History for themes is much improved

- You can only have a single enabled theme. Themes can be selected by
    users, if you opt for it.

On a technical level this change comes with a whole bunch of goodies

- All CSS is now compiled using a custom pipeline that uses libsass
    see /lib/stylesheet

- There is a single pipeline for css compilation (in the past we used
    one for customizations and another one for the rest of the app

- The stylesheet pipeline is now divorced of sprockets, there is no
   reliance on sprockets for CSS bundling

- CSS is generated with source maps everywhere (including themes) this
    makes debugging much easier

- Our "live reloader" is smarter and avoid a flash of unstyled content
   we run a file watcher in "puma" in dev so you no longer need to run
   rake autospec to watch for CSS changes
2017-04-12 10:53:49 -04:00
Guo Xiang Tan 24d3c77d19 Fix the build. 2017-04-12 10:38:17 -04:00
Guo Xiang Tan 3861bd2793 FIX: Quotes should be ignored when parsing for onebox source. 2017-04-11 15:22:21 +08:00
Guo Xiang Tan 0a4c30bce3 FIX: Handle cases where `alt` and `title` tag is blank when parsing excerpt. 2017-04-11 14:18:27 +08:00
Guo Xiang Tan e49f3a408e FEATURE: Add option for `ExcerptParser` to keep onebox source. 2017-04-10 16:11:58 +08:00
Robin Ward 17f2974d0a SECURITY: Confirm new administrator accounts via email 2017-04-04 15:59:01 -04:00
Guo Xiang Tan f4758a4c4d FEATURE: Allow admins to schedule a topic to be published in the future. 2017-04-04 11:16:05 +08:00
Guo Xiang Tan 34b7bee568 FEATURE: Allow admin to auto reopen at topic.
* This commit also introduces a `TopicStatusUpdate`
  model to support other forms of deferred topic
  status update in the future.
2017-03-31 11:14:18 +08:00
Robin Ward 14410b71fb Convert server side paths to use `/u/` 2017-03-30 10:23:24 -04:00
Yana Agun Siswanto cd2d2f16e5 Allow to order search results by the topic creation date
based on: https://meta.discourse.org/t/allow-to-order-search-results-by-the-topic-creation-date/38544
2017-03-30 01:18:38 +07:00
Arpit Jalan 8bf12502bd Merge pull request #4780 from techAPJ/send-statistics
FEATURE: Send anonymized usage statistics to Discourse if "Discourse Hub" can't reach the site
2017-03-28 10:02:05 +05:30
Arpit Jalan f3cd5f61c5 FEATURE: Send anonymized usage statistics to Discourse if Discourse Hub can't reach the site 2017-03-28 09:07:23 +05:30
Régis Hanol dd1cc23caf fix randomly failing specs 2017-03-27 22:47:41 +02:00
Arpit Jalan 3449339fea FIX: admin locales were not getting converted to message format 2017-03-25 01:12:23 +05:30
Guo Xiang Tan b87c5eb1b6 Fix randomly failing spec. 2017-03-24 15:46:42 +08:00
Arpit Jalan 1853a4852c FIX: use email prefix only in subject 2017-03-21 20:29:57 +05:30
Guo Xiang Tan 5169ef8814 Fix broken specs. 2017-03-16 15:05:28 +08:00
Guo Xiang Tan 6312b8b7e4 Fix failing specs. 2017-03-14 14:58:22 +08:00
Guo Xiang Tan 1a7e954e09 FIX: Store custom emojis as uploads.
* Depending on a hardcoded directory was a flawed design
  which made it impossible to debug when custom emojis go
  missing.
2017-03-14 13:07:18 +08:00
Sam 82ca0e368e FEATURE: stop escaping special chars in title prettify
This feature is confusting and just leads to inconsistency
2017-03-13 10:02:20 -04:00
Sam 6ebddc42d1 FIX: include children categories when searching a category 2017-03-10 15:58:47 -05:00
Guo Xiang Tan c623951306 FEATURE: Search can be scoped to posts that the current user has seen/unseen.
https://meta.discourse.org/t/advanced-search-posts-that-i-have-seen/57966
2017-03-09 01:01:33 +08:00
Régis Hanol ee9d621d9c FIX: surround the FROM alias with " in order to support the @ character 2017-03-07 23:37:21 +01:00