e8a6323bea
remove crawler blocking until multisite support
2018-07-03 17:54:45 -04:00
b87fa6d749
FIX: blacklisted crawlers could get through by omitting the accept header
2018-04-17 12:39:30 -04:00
9980f18d86
FEATURE: track request queueing as early as possible
2018-04-17 18:06:17 +10:00
7311023a52
Merge pull request #5700 from discourse/crawl-block
...
FEATURE: control web crawlers access with white/blacklist
2018-03-27 15:06:03 -04:00
4d12ff2e8a
when writing cache, remove elements from the user agents list. also return a message and content type when blocking a crawler.
2018-03-27 13:44:14 -04:00
31dea5d5fc
correct flaky spec
2018-03-27 17:57:19 +11:00
a84bb81ab5
only applies to get html requests
2018-03-22 17:57:44 -04:00
ced7e9a691
FEATURE: control which web crawlers can access using a whitelist or blacklist
2018-03-22 15:41:02 -04:00
f0d5f83424
FEATURE: limit assets less that non asset paths
...
By default assets can be requested up to 200 times per 10 seconds
from the app, this includes CSS and avatars
2018-03-06 15:20:39 +11:00
df8e43abdd
use lazy & instead of try
...
unregister ip skipper in test
raise if called when a skipper is in play
2018-02-06 10:38:15 +11:00
eefd226611
Add extensibility point to `request_tracker` to skip IP addresses
...
This is useful if you want to run a per IP rate limiter but want to be
able to skip some IPs with custom logic.
2018-02-05 17:49:40 -05:00
f26ff290c3
FEATURE: Shorten setting name to max_reqs
...
So it is consistent with other settings
2018-01-22 13:18:30 +11:00
d7657d8e47
correct specs, ensure crawler layout only applies to html
2018-01-16 16:28:11 +11:00
cecd7d0d07
FEATURE: global rate limiter can bypass local IPs
2018-01-08 08:39:17 +11:00
4986ebcf24
FEATURE: optional default off global per ip rate limiter
2017-12-11 17:52:57 +11:00
a4c539bade
FEATURE: Allow registration of detailed request logger
...
Detailed request loggers can be used to gather rich timing info
from all requests (which in turn can be forwarded to monitoring solution)
Middleware::RequestTracker.detailed_request_logger(->|env, data| do
# do stuff with env and data
end
2017-10-18 12:10:30 +11:00
5012d46cbd
Add rubocop to our build. ( #5004 )
2017-07-28 10:20:09 +09:00
3e50313fdc
Prepare for separation of RSpec helper files
...
Since rspec-rails 3, the default installation creates two helper files:
* `spec_helper.rb`
* `rails_helper.rb`
`spec_helper.rb` is intended as a way of running specs that do not
require Rails, whereas `rails_helper.rb` loads Rails (as Discourse's
current `spec_helper.rb` does).
For more information:
https://www.relishapp.com/rspec/rspec-rails/docs/upgrade#default-helper-files
In this commit, I've simply replaced all instances of `spec_helper` with
`rails_helper`, and renamed the original `spec_helper.rb`.
This brings the Discourse project closer to the standard usage of RSpec
in a Rails app.
At present, every spec relies on loading Rails, but there are likely
many that don't need to. In a future pull request, I hope to introduce a
separate, minimal `spec_helper.rb` which can be used in tests which
don't rely on Rails.
2015-12-01 20:39:42 +00:00
86cd1a19cc
FEATURE: page view stats for mobile view
2015-07-03 17:19:33 -04:00
b8cbe51026
Convert specs to RSpec 2.99.2 syntax with Transpec
...
This conversion is done by Transpec 3.1.0 with the following command:
transpec
* 424 conversions
from: obj.should
to: expect(obj).to
* 325 conversions
from: == expected
to: eq(expected)
* 38 conversions
from: obj.should_not
to: expect(obj).not_to
* 15 conversions
from: =~ /pattern/
to: match(/pattern/)
* 9 conversions
from: it { should ... }
to: it { is_expected.to ... }
* 5 conversions
from: lambda { }.should_not
to: expect { }.not_to
* 4 conversions
from: lambda { }.should
to: expect { }.to
* 2 conversions
from: -> { }.should
to: expect { }.to
* 2 conversions
from: -> { }.should_not
to: expect { }.not_to
* 1 conversion
from: === expected
to: be === expected
* 1 conversion
from: =~ [1, 2]
to: match_array([1, 2])
For more details: https://github.com/yujinakayama/transpec#supported-conversions
2015-04-25 11:18:35 -04:00
cbe18eb0df
FEATURE: allow view exclusion using custom header
...
Set Discourse-Track-View to either "0" or "false" to exclude request
2015-02-26 11:41:11 +11:00
acda6ebd60
FIX: view tracking needs to release data earlier
...
retaining data during queuing was causing huge memory spikes
2015-02-10 17:03:33 +11:00
820ce8765e
refactor traffic report
...
split traffic report in 2, page view vs raw traffic
hide raw traffic report by default
improve flushing logic for application reqs
2015-02-06 14:39:16 +11:00
08b790b3c2
improve metrics gathered using in our traffic section
...
this also pulls out the middleware into its own home and inserts in front
2015-02-05 16:08:52 +11:00
Neil Lalonde
Sam
Robin Ward
Guo Xiang Tan
Andy Waite
Arthur Neves