Commit Graph

91 Commits

Author SHA1 Message Date
Jacob Mischka f9247dabcc
Replace `-depth` -> `-maxdepth` in boot_dev (#14046)
The `-depth` flag is incorrect on Linux, it does not take an argument
and causes an error and results in no plugins ever being found.

Copied from `man find`:

```
The global options occur after the list of start points, and so are not the same kind of option as -L, for example.

       -d     A synonym for -depth, for compatibility with FreeBSD, NetBSD, MacOS X and OpenBSD.

       -depth Process each directory's contents before the directory itself.  The -delete action also implies -depth.

       ...

       -maxdepth levels
              Descend at most levels (a non-negative integer) levels of directories below the starting-points.  Using -maxdepth 0  means
              only apply the tests and actions to the starting-points themselves.
```
2021-08-16 13:28:54 +08:00
David Taylor 2955d64703
DEV: Allow annotations to work in symlinked plugins, add binstub (#13635) 2021-07-05 15:43:10 +01:00
Penar Musaraj fbe004cc63
DEV: Do not add proxy argument when running ember-cli test (#13498) 2021-06-23 10:03:52 -04:00
David Taylor 3b6d6c7024
DEV: Set DISCOURSE_PORT when spawning unicorn via `ember-cli -u` (#13346)
This means that Discourse will use the ember-cli proxy's port number in various places like auth redirects and emails
2021-06-09 15:32:28 +01:00
Penar Musaraj 513bfc3a6c
DEV: bin/ember-cli standalone by default (#13344) 2021-06-09 09:48:43 -04:00
Sam 0241748876
DEV: ember-cli -u can be used to run a standalone dev discourse (#13336)
Previously we would need to launch unicorn separately this achieves
the same goal by making 2 modifications:

1. If -u is supplied ember-cli binary will launch and monitor ember cli and unicorn
2. We suppress 200 requests to keep console clean (we may consider moving to development rails logs)


Also cleans out output a bit by supplying silent flags to yarn.
2021-06-09 12:44:33 +10:00
David Taylor 78f9d47ab1 DEV: Add non-x86_64 warning to `d/boot_dev`
Running a development environment using Docker's qemu architecture emulation is currently not possible because `inotify` is not supported: https://github.com/docker/for-mac/issues/5321
2021-05-21 16:51:10 +01:00
David Taylor 73050d9049 DEV: Update docker development binstubs
- Add `d/ember-cli`, and publish port 4200
- Remove `d/sidekiq`. Sidekiq is now started with the rails server
- Move all `docker exec` logic into a single place, so we have one place to set environment variable pass-throughs
- Use `exec` for all bash scripts, so that return statuses are passed back correctly
- Avoid using `bin/bash -c` unnecessarily, because it makes escaping arguments difficult
2021-05-21 16:51:10 +01:00
David Taylor ab23412113 DEV: Only mount top-level plugin symlinks
Sometimes plugins directories will end up with other symlinks (e.g. inside node_modules folders). This logic does not work with deeply nested symlinks, and they are unlikely to be necessary for the plugin to work. Therefore we should only look for symlinks in the top-level of the `plugins` directory
2021-05-21 16:51:10 +01:00
Robin Ward f165f98cf7
DEV: Run `yarn install` when running `bin/ember-cli` (#13102)
Some people have noticed that if we change the packages in package.json
that they have to manually run `yarn install` or Discourse won't work.

This adds `yarn install` to the `bin/ember-cli` helper we run. It seems
quite fast if there is nothing to install so it shouldn't hurt to do
this every time we start the server.
2021-05-20 14:30:22 -04:00
Gerhard Schlager 4c26dd09e4
DEV: Update `bin/bundle` (#13067)
This fixes the following error I've been seeing lately in RubyMine:

> Error:Your `bin/bundle` was not generated by Bundler, so this binstub cannot run.
> Replace `bin/bundle` by running `bundle binstubs bundler --force`, then run this command again.
2021-05-14 19:54:10 +02:00
David Taylor b6b27bc383
DEV: Improve auto-restart parent process detection logic (#13068)
The auto restart logic was sending a USR2 to the parent process without checking what the parent process actually was. In some situations, it might not be the `bin/unicorn` supervisor.

This commit switches to use a global variable for the supervisor PID. This will be much less prone to unexpected behavior.
2021-05-14 18:17:31 +01:00
Jeff Wong 7a0e5c8cb9
FIX: properly filter ember-cli args (#13011)
Was previously rejecting all args.
2021-05-10 10:39:54 -07:00
Penar Musaraj 49de5dadf7
DEV: Ember CLI postinstall hook and shortcut (#12987) 2021-05-10 09:34:24 -04:00
Jarek Radosz 53c06442ef
DEV: Clean up .gitignore (#12981)
(inspired by martin-brennan's recent adventure with `tags` exclusion)

**tl;dr: some of it is obsolete, some is dev-env specific and should live in *your* ~/.gitignore.**

---

To enable global gitignore (`~/.gitignore`) run:
```bash
git config --global core.excludesFile '~/.gitignore'
```

Then create that file and add your VIM/VSCode/Sublime/Emacs/Eclipse/RubyMine/JetBrains/macOS/Arch/Windows 95-specific stuff.

---

Reasons for removal:

* `bin` - never really ignored, all the files are checked in
* `.DS_Store`, `._.DS_Store` - OS specific
* `.sass-cache/*`, `/.bundle`, `/bundle/*`, `/cache`, `/logfile`, `!/plugins/discourse-nginx-performance-report`, `MiniProfiler/Ruby/rack-mini-profiler-2.0.1a.gem`, `config/fog_credentials.yml`, `/public/stylesheet-cache/*`, `script/download_db`, `script/refresh_db`, `config/version.rb` - no longer used?
* `/log/*.log` - covered by /log
* `bootsnap-load-path-cache`, `bootsnap-compile-cache` - bootsnap now keeps its ~~trash~~ temporary files in `/tmp`
* `/.project`, `/.buildpath`, `/.byebug_history`, `/.idea`, `discourse.sublime-workspace`, `*~`, `*.swp`, `*.swo`, `*.swm`, `config/multisite1.yml`, `.rvmrc`, `.ruby-version`, `.ruby-gemset`, `.rbenv`, `bundler_stubs/*`, `*.db`, `*.iml`, `*.swn`, `/package-lock.json`, `.vscode`, `.envrc`, `tags` - dev-env specific, see the note at the top of the commit
2021-05-10 13:43:13 +02:00
Jeff Wong bc3c3d56e0
DEV: do not cd into yarn dir, use --cwd argument to yarn instead (#12986)
* DEV: do not cd into yarn dir, use --cwd argument to yarn instead

Co-authored-by: Robin Ward <robin.ward@gmail.com>
2021-05-07 09:58:57 -07:00
David Taylor ac1d344368
DEV: Update server startup warning to be more specific (#12949)
Using `bundle exec` will slow down server startup by at least 0.5s. `bin/unicorn` has built-in handling of bundler dependencies, so it is better to launch `bin/rails s` or `bin/unicorn` directly.
2021-05-05 13:33:11 +01:00
David Taylor 0188d53f3a
DEV: Add performance warning message when running `rails s` (#12912)
In Discourse, `rails s` ultimately launches the `bin/unicorn` script. However, the overhead of `rails` launching `bin/rails`, and then in turn `bin/unicorn` can be non-trivial. Therefore it is much better to run `bin/unicorn` directly.

This commit prints a warning message to STDERR when `rails s` is used. Functionality is unchanged.
2021-04-30 11:50:06 +01:00
Robin Ward 51f872f13a
DEV: Require Ember CLI to be used in development mode (#12738)
We really want to encourage all developers to use Ember CLI for local
development and testing. This will display an error page if they are not
with instructions on how to start the local server.

To disable it, you can set `NO_EMBER_CLI=1` as an ENV variable
2021-04-29 14:13:36 -04:00
Sam 307cb1d114
DEV: add helper script to run ember cli (#12005)
This little helper script allows for easy ember cli development.

To see the options run `bin/ember-cli -h`

It allows you to proxy try.discourse.org with the `bin/ember-cli --try`
switch, which effectively allows for some development without a rails installed.

It passes on arguments to ember-cli so you can customize port and so on.

It makes the assumption that on local people are using `bin/unicorn` for
development. (it includes some extra discourse specific helpers)
2021-02-09 16:33:14 +11:00
Arpit Jalan bbf6cc4f18
DEV: require fileutils gem on boot (#11542) 2020-12-21 16:31:06 +11:00
Robin Ward 2eb9c0f3dd FIX: `notify_file_change` was outputting a command to vim
The check for `socat` needs to be silenced too, or vim will output the
path to `socat` every time you save a file.
2020-12-10 10:03:57 -05:00
Alessandro Bertacco dc2f5064e2
DEV: support for relative symlinks for plugins in docker dev (#10955)
These changes gives the possibility to use relative symlinks for plugins. Previously only absolute symlinks was supported.
2020-10-21 09:39:53 +11:00
David Taylor 46d1c91e1a
DEV: Make notify_file_change fallback to `nc` when `socat` missing (#10903) 2020-10-13 17:03:24 +01:00
Kane York 4635be10c8 DEV: Add docker cleanup script to d/ folder 2020-03-01 12:09:07 -08:00
Sam Saffron d8412f409a DEV: resolve symlinks in docker dev
symlinks where not fully resolved leading to docker not booting when you
had symlinks in the plugins directory pointing at relative paths.
2020-01-13 10:33:34 +11:00
Martin Brennan aa2b3e71b4 Minor refactor of bin/unicorn cache clean
* move git ls-files for plugins folder into one line
2019-11-20 08:47:51 +10:00
Sam Saffron 9652fdd111 DEV: if tmp is missing create it
Some scripts and people may be nuking the entire tmp folder
2019-11-20 09:29:34 +11:00
Martin Brennan 93d7abe372
FIX: Automatically recover from bad sprockets cache in development (#8364)
We were having issues in development mode where the JS code had errors due to a bad cache. When starting a server in development mode in bin/unicorn we now get the git sha of the discourse HEAD and get a git sha of all plugins, and store them in a file. If the sha has changed then we delete tmp/cache to refresh the assets cache.
2019-11-19 09:15:09 +10:00
Sam Saffron ff33899323 FEATURE: allow publishing docker dev either locally or net wide
Previously we had no control over how internal ports in the containter got
published.

Following UNICORN_BIND_ALL=true setting this broke docker dev env and exposed
this weakness.

The new `d/boot_dev` will only export on localhost, if you wish to export
network with use `d/boot_dev -p`
2019-11-04 12:51:35 +11:00
Mark VanLandingham 9b4aba0d39
DEV: support --fail-fast in bin/turbo_rspec (#8170)
* [WIP] - default turbo spec env to test

* FEATURE: support for --fast-fail in bin/turbo_rspec

* fast-fail -> fail_fast to match rspec

* Moved thread killing outside of fail-fast check

* Removed failure_count incrementation from fast_fail_met
2019-10-09 09:40:06 -05:00
Sam Saffron 7c084c7cab DEV: allow USR2 to restart unicorn in dev
Well all this does is amends a commit comment cause I was over eager and
pushed previous commit.

This is the comment I wanted....

This allows `pkill -USR2 -f 'ruby bin/unicorn'` to restart current unicorn

It is handy if you want to bind a keyboard shortcut for unicorn restarts
in dev.

By doing so you can avoid finding the terminal, hitting ctrl-c and then
hitting up, enter, heading back to browser.

It saves time.

Automate stuff, you will not regret it...
2019-09-11 17:45:47 +10:00
Sam Saffron fb8d1f35a4 DEV: support USR2 to restart unicorn in dev mode
In dev mode sending USR2 to the unicorn master supervisor process will
restart unicorn.

This allows a simple script like this to restart unicorn in dev:

```
#!/usr/bin/env bash

kill -s USR2 `ps aux | grep ruby\ bin\/unicorn | grep -v grep | awk '{print $2}'`
```
2019-09-11 17:35:58 +10:00
hawm 16681cb648 Fix docker image name (#8058) 2019-09-04 16:19:48 +10:00
Kyle Zhao 2fbafd077c DEV: Mount plugin symlinks to dev docker container (#8002)
When developing using docker, in order to support symlinks in the
`plugins/` directory, this reads the symlinks' values and mounts them to
the dev docker container.
2019-08-14 18:13:01 +10:00
Kyle Zhao e2df331fdb DEV: default to test env when running d/rspec (#7962) 2019-08-02 16:40:46 +10:00
Daniel Waterworth c3db5925a8 FIX: Turbo tests exit codes 2019-07-09 08:51:23 +01:00
Daniel Waterworth d6aa92e98e DEV: Add a verbose option to ./bin/turbo_rspec 2019-06-27 15:49:21 +01:00
Sam Saffron fc84e23b71 DEV: allow bin/turbo_tests to run tests without params 2019-06-21 11:33:22 +10:00
Daniel Waterworth e18ce56f4b DEV: Add a new way to run specs in parallel with better output (#7778)
* DEV: Add a new way to run specs in parallel with better output

This commit:

 1. adds a new executable, `bin/interleaved_rspec` which works much like
    `rspec`, but runs the tests in parallel.

 2. adds a rake task, `rake interleaved:spec` which runs the whole test
    suite.

 3. makes autospec use this new wrapper by default. You can disable this
    by running `PARALLEL_SPEC=0 rake autospec`.

It works much like the `parallel_tests` gem (and relies on it), but
makes each subprocess use a machine-readable formatter and parses this
output in order to provide a better overall summary.

(It's called interleaved, because parallel was taken and naming is
hard).

* Make popen3 invocation safer

* Use FileUtils instead of shelling out

* DRY up reporter

* Moved summary logic into Reporter

* s/interleaved/turbo/g

* Move Reporter into its own file

* Moved run into its own class

* Moved Runner into its own file

* Move JsonRowsFormatter under TurboTests

* Join on threads at the end

* Acted on feedback from eviltrout
2019-06-21 10:59:01 +10:00
Sam Saffron 30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Sam Saffron 12e78d8915 DEV: add wrapper for rspec running in docker 2019-05-05 13:43:45 +10:00
Sam Saffron 3b77fb1fb1 DEV: support symlinked plugins in bin/rake autospec
Previously autospec would not pick up save if you saved a plugin in a
symlinked path, this broke quite a few workflows

We now maintain a reverse map so we can correctly re-run specs in plugins
2019-05-03 13:51:07 +10:00
Sam 0e6d0b0f8e FEATURE: always update docker image for dev prior to launching
Previously people could stay stuck on old dev images, this ensures they
are always on latest when booting
2019-02-18 13:12:00 +11:00
Sam 384135845b FEATURE: introduce ultra_low priority queue
This commit introduces an ultra low priority queue for post rebakes. This
way rebakes can never interfere with regular sidekiq processing for cases
where we perform a large scale rebake.

Additionally it allows Post.rebake_old to be run with rate_limiter: false
to avoid triggering the limiter when rebaking. This is handy for cases
where you want to just force the full rebake and not wait for it to trickle
2019-01-17 14:53:19 +11:00
Sam 77d947701c DEV: Add missing discourse script to docker dev 2019-01-07 13:10:37 +11:00
Sam fef45789c4 DEV: improve docker dev environment
- pass USER=discourse which is not passed in from docker exec (improves backup restore)

- add unicorn script and unicorn support (expose port 9292)
2018-12-31 10:45:08 +11:00
Sam 6cf2e64e44 DEV: always run migrations for plugins in dev
Otherwise devs forget to run plugin migrations and get confused when test
does not work. We ship critical migrations in the bundled polls plugin.
2018-12-06 17:19:46 +11:00
Rishabh 10723ba1c2
Remove Docker dev-install instructions
To have a single source of truth for these install instructions in here:
https://meta.discourse.org/t/beginners-guide-to-install-discourse-for-development-using-docker/102009
2018-11-16 15:17:21 +05:30
Sam f9f490e957 DEV: bin/unicorn -x to run without sidekiq 2018-08-17 10:01:44 +10:00