Commit Graph

2777 Commits

Author SHA1 Message Date
Roman Rizzi f5c707c97a
FEATURE: Gz to zip for exports (#7889)
* Revert "Revert "FEATURE: admin/user exports are compressed using the zip format (#7784)""

This reverts commit f89bd55576.

* Replace .tar.zip with .zip
2019-07-18 09:34:48 -03:00
Gerhard Schlager 9a11a8b33b FEATURE: Site setting for typographic quotation marks
Adds locale defaults for German and French
2019-07-11 23:19:28 +02:00
romanrizzi f89bd55576 Revert "FEATURE: admin/user exports are compressed using the zip format (#7784)"
This reverts commit 8b2580e20f.
2019-07-10 11:38:51 -03:00
Roman Rizzi 8b2580e20f
FEATURE: admin/user exports are compressed using the zip format (#7784)
* FEATURE: admin/user exports are compressed using the zip format

* Update translations. Theme exporter now exports .zip file. Theme importer supports .zip and .gz files

* Fix controller test, updated locale and skip saving the csv export to disk
2019-07-10 11:13:03 -03:00
Penar Musaraj 7b0517895e FEATURE: Add "Group owners" to posting options for groups
Context: https://meta.discourse.org/t/121589

This new setting option lets group owners message/mention large groups
without granting that privilege to all members.
2019-07-08 17:14:11 -04:00
Penar Musaraj b690fc3d98
FEATURE: Add new group visibility option for "logged on users" (#7814)
Groups can now be marked as visible to "logged on users". All automatic groups (except `everyone`) are now visible to "logged on users", previously they were marked as public but suppressed in the group page for non-staff.
2019-07-08 15:09:50 -04:00
Robin Ward a075fd46fd FIX: Don't use exceptions to catch conflicts
If a database exception is raised ActiveRecord will always rollback
even if caught.

Instead we build the query in manual SQL and DO NOTHING when there's a
conflict. If we detect nothing was done, perform an update.
2019-07-06 14:43:56 -04:00
Robin Ward c2c169f5b7 FIX: Fail if none of our tags could be updated
For example, if a category has a tag restriction and the API tries to
attempt to update it but cannot.

See:
https://meta.discourse.org/t/unallowed-tag-in-conversation-returns-200/122170
2019-07-05 11:40:18 -04:00
Osama Sayegh 65a6f3080e FIX: don't disable download_remote_images_to_local if site uses S3 (#7861) 2019-07-05 13:36:03 +10:00
Robin Ward 72bac61c90 FIX: Upsert a custom field if a unique constraint fails 2019-07-04 13:26:25 -04:00
Penar Musaraj 03805e5a76
FIX: Ensure lightbox image download has correct content disposition in S3 (#7845) 2019-07-04 11:32:51 -04:00
Josh Moore 6c5689984f FEATURE: in:tagged search (srv side) (#7822)
* FEATURE: in:tagged and in:untagged advanced search filters

Similar to in:solved or in:unsolved, the filters check for an
existence of the topic_id in the topic_tags table.

see: https://meta.discourse.org/t/how-to-search-filter-untagged-topics/119641/2
2019-06-28 18:19:57 +10:00
Sam Saffron 8f7a387aa7 FEATURE: add support for tag group search
The behaviour of #TERM in search has been amended

1. We try category or subcategory slugs
2. We try tags
3. We try tag-groups

The term `hello #my-group` will search for all posts tagged with any of
the tags in the tag group `My Group`

Future work may be introducing a slug cache here or caching it in the table
but the assumption is that the number of tag groups will not be huge
2019-06-27 17:53:26 +10:00
Jeff Wong 88ef5e55fe
FEATURE: add ability to have multiple totp factors (#7626)
Adds a second factor landing page that centralizes a user's second factor configuration.

This contains both TOTP and Backup, and also allows multiple TOTP tokens to be registered and organized by a name. Access to this page is authenticated via password, and cached for 30 minutes via a secure session.
2019-06-26 16:58:06 -07:00
Régis Hanol 134a4c66b7 PERF: limit time spent diffing large blobs of text
REFACTOR: s/.length/.size/g
2019-06-27 01:45:52 +02:00
Arpit Jalan 3c64dc3e7d
FIX: do not allow creation of topic if there is no category available for posting (#7786) 2019-06-26 16:32:53 +05:30
Bianca Nenciu 557805249d
FIX: Ensure topic exists before making a banner. (#7781) 2019-06-25 13:49:29 +03:00
David Taylor afb5ec811d FIX: Don't use DistributedCache to store redis readonly state
This can cause unbound CPU usage in some cases, and excessive logging in other cases. This commit moves redis readonly information into the local process, but maintains the DistributedCache for postgres readonly state.
2019-06-25 11:20:34 +08:00
Penar Musaraj f51f37eddf FEATURE: apply a small penalty to closed topics when searching (#7782) 2019-06-21 12:03:45 +10:00
Robin Ward 0d84c5b894 FIX: If a user deletes a hidden post, it should not lose history 2019-06-20 12:38:16 -04:00
Robin Ward e8b9f38374 FIX: Don't allow users to edit topic information when the OP is locked
see:
https://meta.discourse.org/t/user-able-to-edit-title-of-locked-post/104826
2019-06-18 14:22:38 -04:00
Vinoth Kannan 35d6fff69e PERF: use url instead of file key in temporary inventory table. 2019-06-13 22:03:58 +05:30
Guo Xiang Tan 42ab016856 FIX: Use markdown for images and attachments in `Email::Receiver`. 2019-06-11 14:49:46 +08:00
Guo Xiang Tan 06d974d55c FEATURE: Add base62 sha1 to cooked data attribute
* FEATURE: Add base62 sha1 to data attribute in `Post#cooked`.

* FIX: Use `Upload#short_url` when quoting an image.
2019-06-11 11:15:45 +10:00
Guo Xiang Tan 799bd62803 DEV: Improve `PrettyText` spec to test for markdown image title attr. 2019-06-10 11:00:23 +08:00
Vinoth Kannan 45aebd00a5 SPEC: improve the spec using stubbed S3 client.
4d1204b5e8
2019-06-08 18:10:35 +05:30
Neil Lalonde a08b2589d4 FIX: removing hidden tag bumps topic when all tags are removed
JS sends empty string to remove all tags.
2019-06-07 14:25:46 -04:00
David Taylor e3a9a2d2dd FIX: Avoid infinite loop if disk space is low
We now continue to enqueue the pull_hotlinked_images job for optimized images, even if disk space is low
2019-06-07 14:24:22 +01:00
David Taylor 65b0cafc03 FIX: Always schedule pull_hotlinked_images in cooked_post_processor
The job is now used to pull optimized images, and images from other sites on the same CDN. This needs to run even if download_remote_images is false
2019-06-07 13:08:23 +01:00
Dan Ungureanu 8bd815dab2 FIX: Permit new tags when allow_global_tags true. (#7722) 2019-06-07 15:45:16 +10:00
Régis Hanol b1c5ea4289 FIX: use URI.regexp to find URLs in plain text 2019-06-07 01:26:06 +02:00
Vinoth Kannan b7830680b6 DEV: use cdn url to download the external uploads to local. 2019-06-06 19:17:19 +05:30
Penar Musaraj f00275ded3 FEATURE: Support private attachments when using S3 storage (#7677)
* Support private uploads in S3
* Use localStore for local avatars
* Add job to update private upload ACL on S3
* Test multisite paths
* update ACL for private uploads in migrate_to_s3 task
2019-06-06 13:27:24 +10:00
Gerhard Schlager a77d8bee68 FIX: Use correct locale when translating without cache
follow-up to 6d8eb9c1
2019-06-05 14:19:56 +02:00
Vinoth Kannan 4d1204b5e8 SPEC: Add bucket folder path only if not exists
be0555cc17
2019-06-05 17:21:51 +05:30
Sam Saffron 62141b6316 FEATURE: enable_performance_http_headers for performance diagnostics
This adds support for DISCOURSE_ENABLE_PERFORMANCE_HTTP_HEADERS
when set to `true` this will turn on performance related headers

```text
X-Redis-Calls: 10     # number of redis calls
X-Redis-Time: 1.02    # redis time in seconds
X-Sql-Commands: 102   # number of SQL commands
X-Sql-Time: 1.02      # duration in SQL in seconds
X-Queue-Time: 1.01    # time the request sat in queue (depends on NGINX)
```

To get queue time NGINX must provide: HTTP_X_REQUEST_START

We do not recommend you enable this without thinking, it exposes information
about what your page is doing, usually you would only enable this if you
intend to strip off the headers further down the stream in a proxy
2019-06-05 16:08:11 +10:00
Sam Saffron 6d8eb9c144 PERF: bypass slow locale lookups in various cases
Previously as soon as any override was defined we would regress to the slow
path for locale lookups. Additionally if `raise: true` was specified which
rails likes to add in views we would bypass the cache

The new design manages to use the fast path for many more cases
2019-06-05 14:30:25 +10:00
Neil Lalonde 1cf0b549ab FIX: don't send post edit notification when hidden tags are changed
Create a hidden revision so staff can see the changed, but don't send
notifications to non-staff.
2019-06-04 15:48:15 -04:00
Neil Lalonde dd1a034e7b Revert "FIX: don't send post edit notification when hidden tags are changed" 2019-06-04 15:48:15 -04:00
Gerhard Schlager 98a3e61a2d FIX: Remapping URLs didn't affect upload site settings 2019-06-04 18:37:10 +02:00
Régis Hanol 9756e35956 REVERT: FIX: handle clicks counters in quotes
Not quite a full revert of 7696b92c8c that isn't
actually required.
2019-06-04 11:59:44 +02:00
David Taylor 0508546fd2 FIX: Accept github theme urls with a trailing slash 2019-06-04 10:28:36 +01:00
Guo Xiang Tan 6ad58913c8 DEV: Prefer `save!` over `save`. 2019-06-04 16:59:29 +08:00
Maja Komel 87d3b86484 FIX: better error message when user without permissions replies via email 2019-06-04 16:39:41 +08:00
Sam Saffron 6300d978e2 FIX: correctly escape regex url
duplicate link detection could fail in certain cases
2019-06-04 16:21:03 +10:00
David Taylor 5e61893cb2 FIX: Display generic descriptor for associated account with no info 2019-06-03 12:14:02 +01:00
Dan Ungureanu c1e7a1b292 UX: Merge settings related to muted tags. (#7656) 2019-06-03 12:23:23 +10:00
Maja Komel 42809f4d69 FIX: use crawler layout when saving url in Wayback Machine (#7667) 2019-06-03 12:13:32 +10:00
Guo Xiang Tan 0eb8d74284 DEV: Reduce object fabrication in `CookedPostProcessor` spec. 2019-05-30 08:50:10 +08:00
romanrizzi 2fa8df7cd2 New: can_see_groups? method for better perf 2019-05-30 08:45:20 +08:00
Dan Ungureanu 8728850452 FEATURE: Mute topics tagged with both muted and unmuted tags. 2019-05-30 07:58:17 +08:00
Guo Xiang Tan 8858dbb5b8 DEV: Refactor `SystemMessage#create` specs. 2019-05-30 07:56:36 +08:00
Arpit Jalan 23fdaf07f2 FIX: create system message in user selected locale
DEV: refactor `create_from_system_user` to use `create` method
2019-05-29 21:43:43 +05:30
Guo Xiang Tan 8d1b0224ac Fix the build a3938f98f8. 2019-05-29 18:53:31 +08:00
Guo Xiang Tan f0620e7118 FEATURE: Support `[description|attachment](upload://<short-sha>)` in MD take 2.
Previous attempt was missing `post_uploads` records.
2019-05-29 09:26:32 +08:00
Guo Xiang Tan 6cceb72173 DEV: Reduce verbosity in `TopicQuery` spec. 2019-05-29 08:32:10 +08:00
Penar Musaraj 7c9fb95c15 Temporarily revert "FEATURE: Support `[description|attachment](upload://<short-sha>)` in MD. (#7603)"
This reverts commit b1d3c678ca.

We need to make sure post_upload records are correctly stored.
2019-05-28 16:37:01 -04:00
Guo Xiang Tan b1d3c678ca FEATURE: Support `[description|attachment](upload://<short-sha>)` in MD. (#7603) 2019-05-28 11:18:21 -04:00
Penar Musaraj 42818b810e Better handling of custom SVG sprites in themes when using S3 2019-05-28 10:58:19 -04:00
Sam Saffron 3d2c3bd478 FIX: username suggester incorrectly skipping over whitelisted username
SSO uses a special param to username suggester that whitelists a username
due to previous work we amended our lookup logic and started ignoring this
whitelist.

The fix ensures we always respect it, and also improves on the original
implementation that forgot to normalize the username.
2019-05-28 16:48:46 +10:00
Sam Saffron a02a7d66a9 DEV: properly clean up temp files in FileHelper spec
Followup to 74297003 , also noticed other specs were not properly cleaning
up so fixed that as well.
2019-05-28 11:33:08 +10:00
Sam Saffron 7429700389 FIX: ensure we can download maxmind without redis or db config
This also corrects FileHelper.download so it supports "follow_redirect"
correctly (it used to always follow 1 redirect) and adds a `validate_url`
param that will bypass all uri validation if set to false (default is true)
2019-05-28 10:28:57 +10:00
Blake Erickson 373b8a2139 FIX: Consistently handle category param
See https://meta.discourse.org/t/api-post-to-posts-json-inconsistent-between-users/118571
for more info.

This commit removes a 5 year old temporary fix that is no longer needed.

bc1824a6ed (diff-d8c648926664f849aec050757bfcb6f9R72)

The web interface uses category_id when creating a topic so I think we
should unify on category_id when using the api.
2019-05-27 16:39:56 +08:00
Sam Saffron 170c66c190 DEV: correct flaky test, insufficient timeout
In some cases with all the new mitigations the 10ms delay is no longer
enough. Wait up to 1 second here instead.
2019-05-27 17:08:35 +10:00
Guo Xiang Tan f54e4b71b1 DEV: Make `CookedPostProcessor#post_process_images` method private. 2019-05-27 11:28:37 +08:00
Dan Ungureanu da39d66e83
DEV: Improve support for plugins. (#7332)
* DEV: Allow NewPostManager handlers handle PMs.
* DEV: Add custom fields topic option to PostCreator.
* DEV: Add topic_id to serializer data.
* DEV: Wrap topic title from notification item in a span.
2019-05-25 16:53:03 +03:00
Gerhard Schlager e224100023 FIX: Don't convert :) into Emoji when emojis or emoji shurtcuts are disabled 2019-05-22 11:44:39 +02:00
Gerhard Schlager b788948985 FEATURE: English locale with international date formats
Makes en_US the new default locale
2019-05-20 13:47:20 +02:00
Vinoth Kannan 2a7065c505 FIX: skip uploads without etag in s3 inventory check. 2019-05-20 00:09:52 +05:30
Régis Hanol 7696b92c8c FIX: handle clicks counters in full quotes 2019-05-17 14:17:29 +02:00
David Taylor ef660d5a3e FIX: Return consistent character encodings when downloading S3 uploads
Net::HTTP always returns ASCII-8BIT encoding. File.read auto-detects the encoding. This leads to an encoding inconsistency between a fresh download, and a cached download. This commit ensures all downloaded files are treated equally, by always returning the cached version from the filesystem, even during initial download.

One symptom of this problem is during theme exports: https://meta.discourse.org/t/116907

Related ruby ticket: https://bugs.ruby-lang.org/issues/2567
2019-05-17 11:27:00 +01:00
Guo Xiang Tan 148bfc9be5 DEV: Simplify client and server side code to support removing tags.
Follow up to 834c86678f.
2019-05-17 16:39:20 +08:00
Sam Saffron a8fbb19e7c FEATURE: allow a huge number of users to share common prefix
Previously username suggester would give up after 100 attempts at getting
a username and fallback to random string.

This amends the logic so we do all the work of figuring out a good username
in SQL and avoids a large amount of queries in cases where a lot of usernames
were used up.

This corrects an issue on sites with large numbers of anon users
2019-05-16 17:15:16 +10:00
Régis Hanol fd5c5e326f FIX: remove full quote on direct replies when "typographed"
Use the cooked version of the post and the quote to compare their content in
order to take into account the "typographer" option of the markdown pipeline.
2019-05-15 17:49:29 +02:00
Neil Lalonde 6f747c6b71 FIX: don't allow username to be changed to same as password
We were blocking user registrations with same username and password,
but allowing usernames to be changed to be same as password later.
Also disallow names to be the same as password.
2019-05-13 16:43:40 -04:00
Gerhard Schlager 13e54bca3d DEV: Add specs for handling InvalidPluralizationData exception
follow-up to 6d44be51
2019-05-13 22:07:42 +02:00
Gerhard Schlager 73015521e2 FIX: Correctly calculate fallback locale list
* English shouldn't fallback to any other locale
* Calculate fallback for default locale if it isn't English (useful for en_US)
* Reuse the fallback locale list when outputting translations to JavaScript
2019-05-13 09:36:05 +02:00
Sam Saffron 30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Daniel Waterworth 6e11ae06d8 Prefabricate admin in wizard_spec.rb 2019-05-10 08:34:04 -04:00
Daniel Waterworth bc4748571e Moved fabrication of topic to top level in topics_bulk_action_spec.rb 2019-05-10 08:34:04 -04:00
Daniel Waterworth 90c226949c prefabricate admin in search_spec.rb 2019-05-10 08:34:04 -04:00
Daniel Waterworth 6f5fb020d2 Prefabricate users in post_revisor_spec.rb 2019-05-10 08:34:04 -04:00
Daniel Waterworth cbcfc89fac prefabrication of users in flag_query_spec.rb 2019-05-10 08:34:04 -04:00
David Taylor 2c6b595eed FIX: Process image onebox correctly when image is wrapped in a link
The instagram onebox sometimes surrounds the image with an `<a>` tag, which was breaking the aspect ratio logic, and therefore causing posts to change height on load.
2019-05-10 10:02:40 +01:00
Gerhard Schlager 4a1755b780 FIX: Don't create staged users when processing forwarded email fails 2019-05-09 23:47:47 +02:00
Arpit Jalan 9758857a78 FIX: escape Vimeo iframe URL 2019-05-09 23:01:50 +05:30
Guo Xiang Tan 4e91839c97 FIX: Remove default val for colors step if a default theme has been set.
Running through the wizard after a default theme has been configured
will always revert the default theme to the light theme.
2019-05-09 17:25:09 +08:00
Guo Xiang Tan d110f252bb DEV: Remove unsued option from spec. 2019-05-09 17:25:09 +08:00
Sam Saffron 88650a1259 PERF: avoid checking for consecutive replies in test
This check can issue up to 2 queries per post created, we have specific
tests for it so we can avoid.

This also rolls back #4da6ca4d
2019-05-09 13:28:53 +10:00
Daniel Waterworth 4da6ca4d9f PERF: Skip post validation by default when fabricating posts (#7508)
This speeds up tests by 10%
2019-05-09 10:15:38 +10:00
Penar Musaraj a4eb523af6 Track Discourse user agent pageviews as crawler
Since 5bfe051e, Discourse user agents are marked as non-crawlers (to avoid accidental blacklisting). This makes sure pageviews for these agents are tracked as crawler hits.
2019-05-08 10:38:55 -04:00
Penar Musaraj 8f2c442435 Fix tests 2019-05-08 09:58:47 -04:00
Robin Ward 5af7c90bab FEATURE: Hide Reviewable scores, change score filter to Priority
We found score hard to understand. It is still there behind the scenes
for sorting purposes, but it is no longer shown.

You can now filter by minimum priority (low, med, high) instead of
score.
2019-05-07 14:05:23 -04:00
Bianca Nenciu 98a75906c8
FIX: Close topics deleted by users and ensure topic stubs are destroyed. (#7430) 2019-05-07 16:25:52 +03:00
Daniel Waterworth 32e0c4997c Prefabricated posts 2019-05-07 08:39:45 -04:00
Daniel Waterworth d3a59a43a1 Prefabricated anonymous 2019-05-07 08:39:45 -04:00
Daniel Waterworth 033d584275 Prefabricated topic 2019-05-07 08:39:45 -04:00
Daniel Waterworth 3e2f7d800a Replaced create_topic with Fabricate(:topic) 2019-05-07 08:39:45 -04:00
Daniel Waterworth e79421abb7 Prefabricated users 2019-05-07 08:39:45 -04:00
Daniel Waterworth c9dedd7d72 Removed unnecessary fab! 2019-05-07 08:38:55 -04:00