DEV: Absorb onebox gem into core (#12979)

* Move onebox gem in core library

* Update template file path

* Remove warning for onebox gem caching

* Remove onebox version file

* Remove onebox gem

* Add sanitize gem

* Require onebox library in lazy-yt plugin

* Remove onebox web specific code

This code was used in standalone onebox Sinatra application

* Merge Discourse specific AllowlistedGenericOnebox engine in core

* Fix onebox engine filenames to match class name casing

* Move onebox specs from gem into core

* DEV: Rename `response` helper to `onebox_response`

Fixes a naming collision.

* Require rails_helper

* Don't use `before/after(:all)`

* Whitespace

* Remove fakeweb

* Remove poor unit tests

* DEV: Re-add fakeweb, plugins are using it

* Move onebox helpers

* Stub Instagram API

* FIX: Follow additional redirect status codes (#476)

Don’t throw errors if we encounter 303, 307 or 308 HTTP status codes in responses

* Remove an empty file

* DEV: Update the license file

Using the copy from https://choosealicense.com/licenses/gpl-2.0/#

Hopefully this will enable GitHub to show the license UI?

* DEV: Update embedded copyrights

* DEV: Add Onebox copyright notice

* DEV: Add MIT license, convert COPYRIGHT.txt to md

* DEV: Remove an incorrect copyright claim

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
Co-authored-by: jbrw <jamie@goatforce5.org>

COPYRIGHT.md

@@ -0,0 +1,59 @@
+# Legal notice
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or (at
+your option) any later version.
+
+This program is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
+or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program as the file LICENSE.txt; if not, please see
+http://www.gnu.org/licenses/old-licenses/gpl-2.0.txt.
+
+## Trademark
+
+Discourse is a registered trademark of Civilized Discourse Construction Kit.
+
+## Other copyright notices
+
+Discourse includes works under other copyright notices and distributed
+according to the terms of the GNU General Public License or a compatible
+license (where indicated), including:
+
+- Ember.js - Copyright (c) 2020 Yehuda Katz, Tom Dale and Ember.js contributors
+  MIT License
+
+- jQuery - Copyright OpenJS Foundation and other contributors, https://openjsf.org/
+  MIT License
+
+- Rails - Copyright (c) 2005-2021 David Heinemeier Hansson
+  MIT License
+
+- Onebox - Copyright (c) 2013 jzeta
+  MIT License
+
+MIT License:
+
+```
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+```

COPYRIGHT.txt

@@ -1,31 +0,0 @@
-All Discourse code is Copyright 2013 by Civilized Discourse Construction Kit, Inc.
-
-This program is free software; you can redistribute it and/or modify
-it under the terms of the GNU General Public License as published by
-the Free Software Foundation; either version 2 of the License, or (at
-your option) any later version.
-
-This program is distributed in the hope that it will be useful, but
-WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
-or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
-for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program as the file LICENSE.txt; if not, please see
-http://www.gnu.org/licenses/old-licenses/gpl-2.0.txt.
-
-Discourse is a registered trademark of Civilized Discourse Construction Kit.
-
-Discourse includes works under other copyright notices and distributed
-according to the terms of the GNU General Public License or a compatible
-license (where indicated), including:
-
-Javascript
-
-  Ember.js - Copyright (c) 2012-2013 Yehuda Katz, Tom Dale, Charles Jolley and Ember.js contributors
-
-  jQuery - Copyright (c) 2010-2013 John Resig
-
-Ruby
-
-  Rails - Copyright (c) 2005-2013 David Heinemeier Hansson, Rails Core Team contributors (MIT)

Gemfile

@@ -60,8 +60,6 @@ gem 'redis-namespace'
 # better maintained living fork
 gem 'active_model_serializers', '~> 0.8.3'
 
-gem 'onebox'
-
 gem 'http_accept_language', require: false
 
 # Ember related gems need to be pinned cause they control client side
@@ -229,6 +227,8 @@ gem 'sshkey', require: false
 gem 'rchardet', require: false
 gem 'lz4-ruby', require: false, platform: :ruby
 
+gem 'sanitize'
+
 if ENV["IMPORT"] == "1"
   gem 'mysql2'
   gem 'redcarpet'

Gemfile.lock

@@ -277,13 +277,6 @@ GEM
     omniauth-twitter (1.4.0)
       omniauth-oauth (~> 1.1)
       rack
-    onebox (2.2.15)
-      addressable (~> 2.7.0)
-      htmlentities (~> 4.3)
-      multi_json (~> 1.11)
-      mustache
-      nokogiri (~> 1.7)
-      sanitize
     openssl (2.2.0)
     openssl-signature_algorithm (1.1.1)
       openssl (~> 2.0)
@@ -558,7 +551,6 @@ DEPENDENCIES
   omniauth-google-oauth2
   omniauth-oauth2
   omniauth-twitter
-  onebox
   parallel_tests
   pg
   pry-byebug
@@ -589,6 +581,7 @@ DEPENDENCIES
   ruby-prof
   ruby-readability
   rubyzip
+  sanitize
   sassc (= 2.0.1)
   sassc-rails
   seed-fu
@@ -610,4 +603,4 @@ DEPENDENCIES
   yaml-lint
 
 BUNDLED WITH
-   2.2.16
+   2.2.17

lib/onebox.rb

@@ -0,0 +1,55 @@
+# frozen_string_literal: true
+
+require "openssl"
+require "open-uri"
+require "multi_json"
+require "nokogiri"
+require "mustache"
+require "ostruct"
+require "cgi"
+require "net/http"
+require "digest"
+require "sanitize"
+require_relative "onebox/sanitize_config"
+
+module Onebox
+  DEFAULTS = {
+    connect_timeout: 5,
+    timeout: 10,
+    max_download_kb: (10 * 1024), # 10MB
+    load_paths: [File.join(Rails.root, "lib/onebox/templates")],
+    allowed_ports: [80, 443],
+    allowed_schemes: ["http", "https"],
+    sanitize_config: Sanitize::Config::ONEBOX,
+    redirect_limit: 5
+  }
+
+  @@options = DEFAULTS
+
+  def self.preview(url, options = Onebox.options)
+    Preview.new(url, options)
+  end
+
+  def self.check(url, options = Onebox.options)
+    StatusCheck.new(url, options)
+  end
+
+  def self.options
+    OpenStruct.new(@@options)
+  end
+
+  def self.has_matcher?(url)
+    !!Matcher.new(url).oneboxed
+  end
+
+  def self.options=(options)
+    @@options = DEFAULTS.merge(options)
+  end
+end
+
+require_relative "onebox/preview"
+require_relative "onebox/status_check"
+require_relative "onebox/matcher"
+require_relative "onebox/engine"
+require_relative "onebox/layout"
+require_relative "onebox/view"

lib/onebox/engine.rb

@@ -0,0 +1,203 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    def self.included(object)
+      object.extend(ClassMethods)
+    end
+
+    def self.engines
+      constants.select do |constant|
+        constant.to_s =~ /Onebox$/
+      end.map(&method(:const_get))
+    end
+
+    def self.all_iframe_origins
+      engines.flat_map { |e| e.iframe_origins }.uniq.compact
+    end
+
+    def self.origins_to_regexes(origins)
+      return /.*/ if origins.include?("*")
+      origins.map do |origin|
+        escaped_origin = Regexp.escape(origin)
+        if origin.start_with?("*.", "https://*.", "http://*.")
+          escaped_origin = escaped_origin.sub("\\*", '\S*')
+        end
+
+        Regexp.new("\\A#{escaped_origin}", 'i')
+      end
+    end
+
+    attr_reader :url, :uri, :options, :timeout
+    attr :errors
+
+    DEFAULT = {}
+
+    def options=(opt)
+      return @options if opt.nil? # make sure options provided
+      opt = opt.to_h if opt.instance_of?(OpenStruct)
+      @options.merge!(opt)
+      @options
+    end
+
+    def initialize(url, timeout = nil)
+      @errors = {}
+      @options = DEFAULT
+      class_name = self.class.name.split("::").last.to_s
+
+      # Set the engine options extracted from global options.
+      self.options = Onebox.options[class_name] || {}
+
+      @url = url
+      @uri = URI(url)
+      if always_https?
+        @uri.scheme = 'https'
+        @url = @uri.to_s
+      end
+      @timeout = timeout || Onebox.options.timeout
+    end
+
+    # raises error if not defined in onebox engine.
+    # This is the output method for an engine.
+    def to_html
+      fail NoMethodError, "Engines need to implement this method"
+    end
+
+    # Some oneboxes create iframes or other complicated controls. If you're using
+    # a live editor with HTML preview, rendering those complicated controls can
+    # be slow or cause flickering.
+    #
+    # This method allows engines to produce a placeholder such as static image
+    # frame of a video.
+    #
+    # By default it just calls `to_html` unless implemented.
+    def placeholder_html
+      to_html
+    end
+
+    private
+
+    # raises error if not defined in onebox engine
+    # in each onebox, uses either Nokogiri or StandardEmbed to get raw HTML from url
+    def raw
+      fail NoMethodError, "Engines need to implement this method"
+    end
+
+    # raises error if not defined in onebox engine
+    # in each onebox, returns hash of desired onebox content
+    def data
+      fail NoMethodError, "Engines need this method defined"
+    end
+
+    def link
+      ::Onebox::Helpers.uri_encode(@url)
+    end
+
+    def always_https?
+      self.class.always_https?
+    end
+
+    module ClassMethods
+      def ===(other)
+        if other.kind_of?(URI)
+          !!(other.to_s =~ class_variable_get(:@@matcher))
+        else
+          super
+        end
+      end
+
+      def priority
+        100
+      end
+
+      def matches_regexp(r)
+        class_variable_set :@@matcher, r
+      end
+
+      def requires_iframe_origins(*origins)
+        class_variable_set :@@iframe_origins, origins
+      end
+
+      def iframe_origins
+        class_variable_defined?(:@@iframe_origins) ? class_variable_get(:@@iframe_origins) : []
+      end
+
+      # calculates a name for onebox using the class name of engine
+      def onebox_name
+        name.split("::").last.downcase.gsub(/onebox/, "")
+      end
+
+      def always_https
+        @https = true
+      end
+
+      def always_https?
+        defined?(@https) ? @https : false
+      end
+    end
+  end
+end
+
+require_relative "helpers"
+require_relative "layout_support"
+require_relative "file_type_finder"
+require_relative "engine/standard_embed"
+require_relative "engine/html"
+require_relative "engine/json"
+require_relative "engine/amazon_onebox"
+require_relative "engine/github_issue_onebox"
+require_relative "engine/github_blob_onebox"
+require_relative "engine/github_commit_onebox"
+require_relative "engine/github_folder_onebox"
+require_relative "engine/github_gist_onebox"
+require_relative "engine/github_pull_request_onebox"
+require_relative "engine/google_calendar_onebox"
+require_relative "engine/google_docs_onebox"
+require_relative "engine/google_maps_onebox"
+require_relative "engine/google_play_app_onebox"
+require_relative "engine/image_onebox"
+require_relative "engine/video_onebox"
+require_relative "engine/audio_onebox"
+require_relative "engine/stack_exchange_onebox"
+require_relative "engine/twitter_status_onebox"
+require_relative "engine/wikimedia_onebox"
+require_relative "engine/wikipedia_onebox"
+require_relative "engine/youtube_onebox"
+require_relative "engine/youku_onebox"
+require_relative "engine/allowlisted_generic_onebox"
+require_relative "engine/pubmed_onebox"
+require_relative "engine/sound_cloud_onebox"
+require_relative "engine/imgur_onebox"
+require_relative "engine/pastebin_onebox"
+require_relative "engine/slides_onebox"
+require_relative "engine/xkcd_onebox"
+require_relative "engine/giphy_onebox"
+require_relative "engine/gfycat_onebox"
+require_relative "engine/typeform_onebox"
+require_relative "engine/vimeo_onebox"
+require_relative "engine/steam_store_onebox"
+require_relative "engine/sketch_fab_onebox"
+require_relative "engine/audioboom_onebox"
+require_relative "engine/replit_onebox"
+require_relative "engine/asciinema_onebox"
+require_relative "engine/mixcloud_onebox"
+require_relative "engine/band_camp_onebox"
+require_relative "engine/coub_onebox"
+require_relative "engine/flickr_onebox"
+require_relative "engine/flickr_shortened_onebox"
+require_relative "engine/five_hundred_px_onebox"
+require_relative "engine/pdf_onebox"
+require_relative "engine/twitch_clips_onebox"
+require_relative "engine/twitch_stream_onebox"
+require_relative "engine/twitch_video_onebox"
+require_relative "engine/trello_onebox"
+require_relative "engine/cloud_app_onebox"
+require_relative "engine/wistia_onebox"
+require_relative "engine/simplecast_onebox"
+require_relative "engine/instagram_onebox"
+require_relative "engine/gitlab_blob_onebox"
+require_relative "engine/google_photos_onebox"
+require_relative "engine/kaltura_onebox"
+require_relative "engine/reddit_media_onebox"
+require_relative "engine/google_drive_onebox"
+require_relative "engine/facebook_media_onebox"

lib/onebox/engine/allowlisted_generic_onebox.rb

@@ -1,21 +1,267 @@
 # frozen_string_literal: true
 
+require 'htmlentities'
 require "ipaddr"
 
 module Onebox
   module Engine
     class AllowlistedGenericOnebox
+      include Engine
+      include StandardEmbed
+      include LayoutSupport
+
+      def self.priority
+        200
+      end
+
+      # Often using the `html` attribute is not what we want, like for some blogs that
+      # include the entire page HTML. However for some providers like Flickr it allows us
+      # to return gifv and galleries.
+      def self.default_html_providers
+        ['Flickr', 'Meetup']
+      end
+
+      def self.html_providers
+        @html_providers ||= default_html_providers.dup
+      end
+
+      def self.html_providers=(new_provs)
+        @html_providers = new_provs
+      end
+
+      # A re-written URL converts http:// -> https://
+      def self.rewrites
+        @rewrites ||= https_hosts.dup
+      end
+
+      def self.rewrites=(new_list)
+        @rewrites = new_list
+      end
+
+      def self.https_hosts
+        %w(slideshare.net dailymotion.com livestream.com imgur.com flickr.com)
+      end
+
+      def self.host_matches(uri, list)
+        !!list.find { |h| %r((^|\.)#{Regexp.escape(h)}$).match(uri.host) }
+      end
+
+      def self.allowed_twitter_labels
+        ['brand', 'price', 'usd', 'cad', 'reading time', 'likes']
+      end
 
-      # overwrite the allowlist
       def self.===(other)
         other.is_a?(URI) ? (IPAddr.new(other.hostname) rescue nil).nil? : true
       end
 
-      # ensure we're the last engine to be used
+      def to_html
-      def self.priority
+        rewrite_https(generic_html)
-        Float::INFINITY
       end
 
+      def placeholder_html
+        return article_html if is_article?
+        return image_html if is_image?
+        return Onebox::Helpers.video_placeholder_html if is_video? || is_card?
+        return Onebox::Helpers.generic_placeholder_html if is_embedded?
+        to_html
+      end
+
+      def data
+        @data ||= begin
+          html_entities = HTMLEntities.new
+          d = { link: link }.merge(raw)
+
+          if !Onebox::Helpers.blank?(d[:title])
+            d[:title] = html_entities.decode(Onebox::Helpers.truncate(d[:title], 80))
+          end
+
+          d[:description] ||= d[:summary]
+          if !Onebox::Helpers.blank?(d[:description])
+            d[:description] = html_entities.decode(Onebox::Helpers.truncate(d[:description], 250))
+          end
+
+          if !Onebox::Helpers.blank?(d[:site_name])
+            d[:domain] = html_entities.decode(Onebox::Helpers.truncate(d[:site_name], 80))
+          elsif !Onebox::Helpers.blank?(d[:domain])
+            d[:domain] = "http://#{d[:domain]}" unless d[:domain] =~ /^https?:\/\//
+            d[:domain] = URI(d[:domain]).host.to_s.sub(/^www\./, '') rescue nil
+          end
+
+          # prefer secure URLs
+          d[:image] = d[:image_secure_url] || d[:image_url] || d[:thumbnail_url] || d[:image]
+          d[:image] = Onebox::Helpers::get_absolute_image_url(d[:image], @url)
+          d[:image] = Onebox::Helpers::normalize_url_for_output(html_entities.decode(d[:image]))
+          d[:image] = nil if Onebox::Helpers.blank?(d[:image])
+
+          d[:video] = d[:video_secure_url] || d[:video_url] || d[:video]
+          d[:video] = nil if Onebox::Helpers.blank?(d[:video])
+
+          d[:published_time] = d[:article_published_time] unless Onebox::Helpers.blank?(d[:article_published_time])
+          if !Onebox::Helpers.blank?(d[:published_time])
+            d[:article_published_time] = Time.parse(d[:published_time]).strftime("%-d %b %y")
+            d[:article_published_time_title] = Time.parse(d[:published_time]).strftime("%I:%M%p - %d %B %Y")
+          end
+
+          # Twitter labels
+          if !Onebox::Helpers.blank?(d[:label1]) && !Onebox::Helpers.blank?(d[:data1]) && !!AllowlistedGenericOnebox.allowed_twitter_labels.find { |l| d[:label1] =~ /#{l}/i }
+            d[:label_1] = Onebox::Helpers.truncate(d[:label1])
+            d[:data_1]  = Onebox::Helpers.truncate(d[:data1])
+          end
+          if !Onebox::Helpers.blank?(d[:label2]) && !Onebox::Helpers.blank?(d[:data2]) && !!AllowlistedGenericOnebox.allowed_twitter_labels.find { |l| d[:label2] =~ /#{l}/i }
+            unless Onebox::Helpers.blank?(d[:label_1])
+              d[:label_2] = Onebox::Helpers.truncate(d[:label2])
+              d[:data_2]  = Onebox::Helpers.truncate(d[:data2])
+            else
+              d[:label_1] = Onebox::Helpers.truncate(d[:label2])
+              d[:data_1]  = Onebox::Helpers.truncate(d[:data2])
+            end
+          end
+
+          if Onebox::Helpers.blank?(d[:label_1]) && !Onebox::Helpers.blank?(d[:price_amount]) && !Onebox::Helpers.blank?(d[:price_currency])
+            d[:label_1] = "Price"
+            d[:data_1] = Onebox::Helpers.truncate("#{d[:price_currency].strip} #{d[:price_amount].strip}")
+          end
+
+          skip_missing_tags = [:video]
+          d.each do |k, v|
+            next if skip_missing_tags.include?(k)
+            if v == nil || v == ''
+              errors[k] ||= []
+              errors[k] << 'is blank'
+            end
+          end
+
+          d
+        end
+      end
+
+      private
+
+      def rewrite_https(html)
+        return unless html
+        if AllowlistedGenericOnebox.host_matches(uri, AllowlistedGenericOnebox.rewrites)
+          html = html.gsub("http://", "https://")
+        end
+        html
+      end
+
+      def generic_html
+        return article_html  if is_article?
+        return video_html    if is_video?
+        return image_html    if is_image?
+        return embedded_html if is_embedded?
+        return card_html     if is_card?
+        return article_html  if (has_text? || is_image_article?)
+      end
+
+      def is_card?
+        data[:card] == 'player' &&
+          data[:player] =~ URI::regexp &&
+          options[:allowed_iframe_regexes]&.any? { |r| data[:player] =~ r }
+      end
+
+      def is_article?
+        (data[:type] =~ /article/ || data[:asset_type] =~ /article/) &&
+        has_text?
+      end
+
+      def has_text?
+        has_title? && !Onebox::Helpers.blank?(data[:description])
+      end
+
+      def has_title?
+        !Onebox::Helpers.blank?(data[:title])
+      end
+
+      def is_image_article?
+        has_title? && has_image?
+      end
+
+      def is_image?
+        data[:type] =~ /photo|image/ &&
+        data[:type] !~ /photostream/ &&
+        has_image?
+      end
+
+      def has_image?
+        !Onebox::Helpers.blank?(data[:image])
+      end
+
+      def is_video?
+        data[:type] =~ /^video[\/\.]/ &&
+          data[:video_type] == "video/mp4" && # Many sites include 'videos' with text/html types (i.e. iframes)
+          !Onebox::Helpers.blank?(data[:video])
+      end
+
+      def is_embedded?
+        return false unless data[:html] && data[:height]
+        return true if AllowlistedGenericOnebox.html_providers.include?(data[:provider_name])
+        return false unless data[:html]["iframe"]
+
+        fragment = Nokogiri::HTML5::fragment(data[:html])
+        src = fragment.at_css('iframe')&.[]("src")
+        options[:allowed_iframe_regexes]&.any? { |r| src =~ r }
+      end
+
+      def card_html
+        escaped_url = ::Onebox::Helpers.normalize_url_for_output(data[:player])
+
+        <<~RAW
+        <iframe src="#{escaped_url}"
+                width="#{data[:player_width] || "100%"}"
+                height="#{data[:player_height]}"
+                scrolling="no"
+                frameborder="0">
+        </iframe>
+        RAW
+      end
+
+      def article_html
+        layout.to_html
+      end
+
+      def image_html
+        return if Onebox::Helpers.blank?(data[:image])
+
+        escaped_src = ::Onebox::Helpers.normalize_url_for_output(data[:image])
+
+        alt    = data[:description]  || data[:title]
+        width  = data[:image_width]  || data[:thumbnail_width]  || data[:width]
+        height = data[:image_height] || data[:thumbnail_height] || data[:height]
+
+        "<img src='#{escaped_src}' alt='#{alt}' width='#{width}' height='#{height}' class='onebox'>"
+      end
+
+      def video_html
+        escaped_video_src = ::Onebox::Helpers.normalize_url_for_output(data[:video])
+        escaped_image_src = ::Onebox::Helpers.normalize_url_for_output(data[:image])
+
+        <<-HTML
+          <video
+            title='#{data[:title]}'
+            width='#{data[:video_width]}'
+            height='#{data[:video_height]}'
+            style='max-width:100%'
+            poster='#{escaped_image_src}'
+            controls=''
+          >
+            <source src='#{escaped_video_src}'>
+          </video>
+        HTML
+      end
+
+      def embedded_html
+        fragment = Nokogiri::HTML5::fragment(data[:html])
+        fragment.css("img").each { |img| img["class"] = "thumbnail" }
+        if iframe = fragment.at_css("iframe")
+          iframe.remove_attribute("style")
+          iframe["width"] = data[:width] || "100%"
+          iframe["height"] = data[:height]
+          iframe["scrolling"] = "no"
+          iframe["frameborder"] = "0"
+        end
+        fragment.to_html
+      end
     end
   end
 end

lib/onebox/engine/amazon_onebox.rb

@@ -0,0 +1,198 @@
+# frozen_string_literal: true
+
+require 'json'
+require "onebox/open_graph"
+
+module Onebox
+  module Engine
+    class AmazonOnebox
+      include Engine
+      include LayoutSupport
+      include HTML
+
+      always_https
+      matches_regexp(/^https?:\/\/(?:www\.)?(?:smile\.)?(amazon|amzn)\.(?<tld>com|ca|de|it|es|fr|co\.jp|co\.uk|cn|in|com\.br|com\.mx|nl|pl|sa|sg|se|com\.tr|ae)\//)
+
+      def url
+        @raw ||= nil
+
+        # If possible, fetch the cached HTML body immediately so we can
+        # try to grab the canonical URL from that document,
+        # rather than guess at the best URL structure to use
+        if !@raw && has_cached_body
+          @raw = Onebox::Helpers.fetch_html_doc(@url, http_params, body_cacher)
+        end
+
+        if @raw
+          canonical_link = @raw.at('//link[@rel="canonical"]/@href')
+          return canonical_link.to_s if canonical_link
+        end
+
+        if match && match[:id]
+          id = Addressable::URI.encode_component(match[:id], Addressable::URI::CharacterClasses::PATH)
+          return "https://www.amazon.#{tld}/dp/#{id}"
+        end
+
+        @url
+      end
+
+      def tld
+        @tld ||= @@matcher.match(@url)["tld"]
+      end
+
+      def http_params
+        if @options && @options[:user_agent]
+          { 'User-Agent' => @options[:user_agent] }
+        end
+      end
+
+      private
+
+      def has_cached_body
+        body_cacher&.respond_to?('cache_response_body?') &&
+          body_cacher.cache_response_body?(uri.to_s) &&
+          body_cacher.cached_response_body_exists?(uri.to_s)
+      end
+
+      def match
+        @match ||= @url.match(/(?:d|g)p\/(?:product\/|video\/detail\/)?(?<id>[A-Z0-9]+)(?:\/|\?|$)/mi)
+      end
+
+      def image
+        if (main_image = raw.css("#main-image")) && main_image.any?
+          attributes = main_image.first.attributes
+
+          if attributes["data-a-hires"]
+            return attributes["data-a-hires"].to_s
+          elsif attributes["data-a-dynamic-image"]
+            return ::JSON.parse(attributes["data-a-dynamic-image"].value).keys.first
+          end
+        end
+
+        if (landing_image = raw.css("#landingImage")) && landing_image.any?
+          attributes = landing_image.first.attributes
+
+          if attributes["data-old-hires"]
+            return attributes["data-old-hires"].to_s
+          else
+            return landing_image.first["src"].to_s
+          end
+        end
+
+        if (ebook_image = raw.css("#ebooksImgBlkFront")) && ebook_image.any?
+          ::JSON.parse(ebook_image.first.attributes["data-a-dynamic-image"].value).keys.first
+        end
+      end
+
+      def price
+        # get item price (Amazon markup is inconsistent, deal with it)
+        if raw.css("#priceblock_ourprice .restOfPrice")[0] && raw.css("#priceblock_ourprice .restOfPrice")[0].inner_text
+          "#{raw.css("#priceblock_ourprice .restOfPrice")[0].inner_text}#{raw.css("#priceblock_ourprice .buyingPrice")[0].inner_text}.#{raw.css("#priceblock_ourprice .restOfPrice")[1].inner_text}"
+        elsif raw.css("#priceblock_dealprice") && (dealprice = raw.css("#priceblock_dealprice span")[0])
+          dealprice.inner_text
+        elsif !raw.css("#priceblock_ourprice").inner_text.empty?
+          raw.css("#priceblock_ourprice").inner_text
+        else
+          raw.css(".mediaMatrixListItem.a-active .a-color-price").inner_text
+        end
+      end
+
+      def multiple_authors(authors_xpath)
+        raw
+          .xpath(authors_xpath)
+          .map { |a| a.inner_text.strip }
+          .join(", ")
+      end
+
+      def data
+        og = ::Onebox::OpenGraph.new(raw)
+
+        if raw.at_css('#dp.book_mobile') # printed books
+          title = raw.at("h1#title")&.inner_text
+          authors = raw.at_css('#byline_secondary_view_div') ? multiple_authors("//div[@id='byline_secondary_view_div']//span[@class='a-text-bold']") : raw.at("#byline")&.inner_text
+          rating = raw.at("#averageCustomerReviews_feature_div .a-icon")&.inner_text || raw.at("#cmrsArcLink .a-icon")&.inner_text
+
+          table_xpath = "//div[@id='productDetails_secondary_view_div']//table[@id='productDetails_techSpec_section_1']"
+          isbn = raw.xpath("#{table_xpath}//tr[8]//td").inner_text.strip
+
+          # if ISBN is misplaced or absent it's hard to find out which data is
+          # available and where to find it so just set it all to nil
+          if /^\d(\-?\d){12}$/.match(isbn)
+            publisher = raw.xpath("#{table_xpath}//tr[1]//td").inner_text.strip
+            published = raw.xpath("#{table_xpath}//tr[2]//td").inner_text.strip
+            book_length = raw.xpath("#{table_xpath}//tr[6]//td").inner_text.strip
+          else
+            isbn = publisher = published = book_length = nil
+          end
+
+          result = {
+            link: url,
+            title: title,
+            by_info: authors,
+            image: og.image || image,
+            description: raw.at("#productDescription")&.inner_text,
+            rating: "#{rating}#{', ' if rating && (!isbn&.empty? || !price&.empty?)}",
+            price: price,
+            isbn_asin_text: "ISBN",
+            isbn_asin: isbn,
+            publisher: publisher,
+            published: "#{published}#{', ' if published && !price&.empty?}"
+          }
+
+        elsif raw.at_css('#dp.ebooks_mobile') # ebooks
+          title = raw.at("#ebooksTitle")&.inner_text
+          authors = raw.at_css('#a-popover-mobile-udp-contributor-popover-id') ? multiple_authors("//div[@id='a-popover-mobile-udp-contributor-popover-id']//span[contains(@class,'a-text-bold')]") : (raw.at("#byline")&.inner_text&.strip || raw.at("#bylineInfo")&.inner_text&.strip)
+          rating = raw.at("#averageCustomerReviews_feature_div .a-icon")&.inner_text || raw.at("#cmrsArcLink .a-icon")&.inner_text || raw.at("#acrCustomerReviewLink .a-icon")&.inner_text
+
+          table_xpath = "//div[@id='detailBullets_secondary_view_div']//ul"
+          asin = raw.xpath("#{table_xpath}//li[4]/span/span[2]").inner_text
+
+          # if ASIN is misplaced or absent it's hard to find out which data is
+          # available and where to find it so just set it all to nil
+          if /^[0-9A-Z]{10}$/.match(asin)
+            publisher = raw.xpath("#{table_xpath}//li[2]/span/span[2]").inner_text
+            published = raw.xpath("#{table_xpath}//li[1]/span/span[2]").inner_text
+          else
+            asin = publisher = published = nil
+          end
+
+          result = {
+            link: url,
+            title: title,
+            by_info: authors,
+            image: og.image || image,
+            description: raw.at("#productDescription")&.inner_text,
+            rating: "#{rating}#{', ' if rating && (!asin&.empty? || !price&.empty?)}",
+            price: price,
+            isbn_asin_text: "ASIN",
+            isbn_asin: asin,
+            publisher: publisher,
+            published: "#{published}#{', ' if published && !price&.empty?}"
+          }
+
+        else
+          title = og.title || CGI.unescapeHTML(raw.css("title").inner_text)
+          result = {
+            link: url,
+            title: title,
+            image: og.image || image,
+            price: price
+          }
+
+          result[:by_info] = raw.at("#by-line")
+          result[:by_info] = Onebox::Helpers.clean(result[:by_info].inner_html) if result[:by_info]
+
+          summary = raw.at("#productDescription")
+
+          description = og.description || summary&.inner_text
+          description ||= raw.css("meta[name=description]").first&.[]("content")
+          result[:description] = CGI.unescapeHTML(Onebox::Helpers.truncate(description, 250)) if description
+        end
+
+        result[:price] = nil if result[:price].start_with?("$0") || result[:price] == 0
+
+        result
+      end
+    end
+  end
+end

lib/onebox/engine/asciinema_onebox.rb

@@ -0,0 +1,27 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class AsciinemaOnebox
+      include Engine
+      include StandardEmbed
+
+      always_https
+      matches_regexp(/^https?:\/\/asciinema\.org\/a\/[\p{Alnum}_\-]+$/)
+
+      def to_html
+        "<script type='text/javascript' src='https://asciinema.org/a/#{match[:asciinema_id]}.js' id='asciicast-#{match[:asciinema_id]}' async></script>"
+      end
+
+      def placeholder_html
+        "<img src='https://asciinema.org/a/#{match[:asciinema_id]}.png'>"
+      end
+
+      private
+
+      def match
+        @match ||= @url.match(/asciinema\.org\/a\/(?<asciinema_id>[\p{Alnum}_\-]+)$/)
+      end
+    end
+  end
+end

lib/onebox/engine/audio_onebox.rb

@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class AudioOnebox
+      include Engine
+
+      matches_regexp(/^(https?:)?\/\/.*\.(mp3|ogg|opus|wav|m4a)(\?.*)?$/i)
+
+      def always_https?
+        AllowlistedGenericOnebox.host_matches(uri, AllowlistedGenericOnebox.https_hosts)
+      end
+
+      def to_html
+        escaped_url = ::Onebox::Helpers.normalize_url_for_output(@url)
+
+        <<-HTML
+          <audio controls #{@options[:disable_media_download_controls] ? 'controlslist="nodownload"' : ""}>
+            <source src="#{escaped_url}">
+            <a href="#{escaped_url}">#{@url}</a>
+          </audio>
+        HTML
+      end
+
+      def placeholder_html
+        ::Onebox::Helpers.audio_placeholder_html
+      end
+    end
+  end
+end

lib/onebox/engine/audioboom_onebox.rb

@@ -0,0 +1,29 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class AudioboomOnebox
+      include Engine
+      include StandardEmbed
+
+      matches_regexp(/^https?:\/\/audioboom\.com\/posts\/\d+/)
+      always_https
+
+      def placeholder_html
+        oembed = get_oembed
+
+        <<-HTML
+          <img
+            src="#{oembed.thumbnail_url}"
+            style="max-width: #{oembed.width}px; max-height: #{oembed.height}px;"
+            #{oembed.title_attr}
+          >
+        HTML
+      end
+
+      def to_html
+        get_oembed.html
+      end
+    end
+  end
+end

lib/onebox/engine/band_camp_onebox.rb

@@ -0,0 +1,35 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class BandCampOnebox
+      include Engine
+      include StandardEmbed
+
+      matches_regexp(/^https?:\/\/.*\.bandcamp\.com\/(album|track)\//)
+      always_https
+      requires_iframe_origins "https://bandcamp.com"
+
+      def placeholder_html
+        og = get_opengraph
+        "<img src='#{og.image}' height='#{og.video_height}' #{og.title_attr}>"
+      end
+
+      def to_html
+        og = get_opengraph
+        escaped_src = og.video_secure_url || og.video
+
+        <<-HTML
+          <iframe
+            src="#{escaped_src}"
+            width="#{og.video_width}"
+            height="#{og.video_height}"
+            scrolling="no"
+            frameborder="0"
+            allowfullscreen
+          ></iframe>
+        HTML
+      end
+    end
+  end
+end

lib/onebox/engine/cloud_app_onebox.rb

@@ -0,0 +1,53 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class CloudAppOnebox
+      include Engine
+      include StandardEmbed
+
+      matches_regexp(/^https?:\/\/cl\.ly/)
+      always_https
+
+      def to_html
+        og = get_opengraph
+
+        if !og.image.nil?
+          image_html(og)
+        elsif og.title.to_s[/\.(mp4|ogv|webm)$/]
+          video_html(og)
+        else
+          link_html(og)
+        end
+      end
+
+      private
+
+      def link_html(og)
+        <<-HTML
+          <a href='#{og.url}' target='_blank' rel='noopener'>
+            #{og.title}
+          </a>
+        HTML
+      end
+
+      def video_html(og)
+        direct_src = ::Onebox::Helpers.normalize_url_for_output("#{og.get(:url)}/#{og.title}")
+
+        <<-HTML
+          <video width='480' height='360' #{og.title_attr} controls loop>
+            <source src='#{direct_src}' type='video/mp4'>
+          </video>
+        HTML
+      end
+
+      def image_html(og)
+        <<-HTML
+          <a href='#{og.url}' target='_blank' class='onebox' rel='noopener'>
+            <img src='#{og.image}' #{og.title_attr} alt='CloudApp' width='480'>
+          </a>
+        HTML
+      end
+    end
+  end
+end

lib/onebox/engine/coub_onebox.rb

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class CoubOnebox
+      include Engine
+      include StandardEmbed
+
+      matches_regexp(/^https?:\/\/coub\.com\/view\//)
+      always_https
+
+      def placeholder_html
+        oembed = get_oembed
+        "<img src='#{oembed.thumbnail_url}' height='#{oembed.thumbnail_height}' width='#{oembed.thumbnail_width}' #{oembed.title_attr}>"
+      end
+
+      def to_html
+        get_oembed.html
+      end
+    end
+  end
+end

lib/onebox/engine/facebook_media_onebox.rb

@@ -0,0 +1,34 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class FacebookMediaOnebox
+      include Engine
+      include StandardEmbed
+
+      matches_regexp(/^https?:\/\/.*\.facebook\.com\/(\w+)\/(videos|\?).*/)
+      always_https
+      requires_iframe_origins "https://www.facebook.com"
+
+      def to_html
+        metadata = get_twitter
+        if metadata.present? && metadata[:card] == "player" && metadata[:player].present?
+          <<-HTML
+            <iframe
+              src="#{metadata[:player]}"
+              width="#{metadata[:player_width]}"
+              height="#{metadata[:player_height]}"
+              scrolling="no"
+              frameborder="0"
+              allowfullscreen
+            ></iframe>
+          HTML
+        else
+          html = Onebox::Engine::AllowlistedGenericOnebox.new(@url, @timeout).to_html
+          return if Onebox::Helpers.blank?(html)
+          html
+        end
+      end
+    end
+  end
+end

lib/onebox/engine/five_hundred_px_onebox.rb

@@ -0,0 +1,18 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class FiveHundredPxOnebox
+      include Engine
+      include StandardEmbed
+
+      matches_regexp(/^https?:\/\/500px\.com\/photo\/\d+\//)
+      always_https
+
+      def to_html
+        og = get_opengraph
+        "<img src='#{og.image}' width='#{og.image_width}' height='#{og.image_height}' class='onebox' #{og.title_attr}>"
+      end
+    end
+  end
+end

lib/onebox/engine/flickr_onebox.rb

@@ -0,0 +1,52 @@
+# frozen_string_literal: true
+
+require_relative './opengraph_image'
+
+module Onebox
+  module Engine
+    class FlickrOnebox
+      include Engine
+      include StandardEmbed
+
+      matches_regexp(/^https?:\/\/www\.flickr\.com\/photos\//)
+      always_https
+
+      def to_html
+        og = get_opengraph
+        return album_html(og) if og.url =~ /\/sets\//
+        return image_html(og) if !og.image.nil?
+        nil
+      end
+
+      private
+
+      def album_html(og)
+        escaped_url = ::Onebox::Helpers.normalize_url_for_output(url)
+        album_title = "[Album] #{og.title}"
+
+        <<-HTML
+          <div class='onebox flickr-album'>
+            <a href='#{escaped_url}' target='_blank' rel='noopener'>
+              <span class='outer-box' style='max-width:#{og.image_width}px'>
+                <span class='inner-box'>
+                  <span class='album-title'>#{album_title}</span>
+                </span>
+              </span>
+              <img src='#{og.secure_image_url}' #{og.title_attr} height='#{og.image_height}' width='#{og.image_width}'>
+            </a>
+          </div>
+        HTML
+      end
+
+      def image_html(og)
+        escaped_url = ::Onebox::Helpers.normalize_url_for_output(url)
+
+        <<-HTML
+          <a href='#{escaped_url}' target='_blank' rel='noopener' class="onebox">
+            <img src='#{og.secure_image_url}' #{og.title_attr} alt='Imgur' height='#{og.image_height}' width='#{og.image_width}'>
+          </a>
+        HTML
+      end
+    end
+  end
+end

lib/onebox/engine/flickr_shortened_onebox.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+require_relative './opengraph_image'
+
+module Onebox
+  module Engine
+    class FlickrShortenedOnebox
+      include Engine
+      include StandardEmbed
+      include OpengraphImage
+
+      matches_regexp(/^https?:\/\/flic\.kr\/p\//)
+      always_https
+    end
+  end
+end

lib/onebox/engine/gfycat_onebox.rb

@@ -0,0 +1,113 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class GfycatOnebox
+      include Engine
+      include JSON
+
+      matches_regexp(/^https?:\/\/gfycat\.com\//)
+      always_https
+
+      # This engine should have priority over AllowlistedGenericOnebox.
+      def self.priority
+        1
+      end
+
+      def to_html
+        <<-HTML
+          <aside class="onebox gfycat">
+            <header class="source">
+              <img src="https://gfycat.com/static/favicons/favicon-96x96.png" class="site-icon" width="64" height="64">
+              <a href="#{data[:url]}" target="_blank" rel="nofollow ugc noopener">Gfycat.com</a>
+            </header>
+
+            <article class="onebox-body">
+              <h4>
+                #{data[:title]} by
+                <a href="https://gfycat.com/@#{data[:author]}" target="_blank" rel="nofollow ugc noopener">
+                  <span>#{data[:author]}</span>
+                </a>
+              </h4>
+
+              <div class="video" style="--aspect-ratio: #{data[:width]}/#{data[:height]}">
+                <video controls loop muted poster="#{data[:posterUrl]}">
+                  <source id="webmSource" src="#{data[:webmUrl]}" type="video/webm">
+                  <source id="mp4Source" src="#{data[:mp4Url]}" type="video/mp4">
+                  <img title="Sorry, your browser doesn't support HTML5 video." src="#{data[:posterUrl]}">
+                </video>
+              </div>
+
+              <p>
+                <span class="label1">#{data[:keywords]}</span>
+              </p>
+            </article>
+
+            <div style="clear: both"></div>
+          </aside>
+        HTML
+      end
+
+      def placeholder_html
+        <<-HTML
+          <a href="#{data[:url]}">
+            <img src="#{data[:posterUrl]}" width="#{data[:width]}" height="#{data[:height]}"><br/>
+            #{data[:name]}
+          </a>
+        HTML
+      end
+
+      private
+
+      def match
+        @match ||= @url.match(/^https?:\/\/gfycat\.com\/(gifs\/detail\/)?(?<name>.+)/)
+      end
+
+      def og_data
+        return @og_data if defined?(@og_data)
+
+        response = Onebox::Helpers.fetch_response(url, redirect_limit: 10) rescue nil
+        page = Nokogiri::HTML(response)
+        script = page.at_css('script[type="application/ld+json"]')
+
+        if json_string = script&.text
+          @og_data = Onebox::Helpers.symbolize_keys(::MultiJson.load(json_string))
+        else
+          @og_data = {}
+        end
+      end
+
+      def data
+        return @data if defined?(@data)
+
+        @data = {
+          name: match[:name],
+          title: og_data[:headline] || 'No Title',
+          author: og_data[:author],
+          url: @url,
+        }
+
+        if keywords = og_data[:keywords]&.split(',')
+          @data[:keywords] = keywords
+            .map { |keyword| "<a href='https://gfycat.com/gifs/search/#{keyword}'>##{keyword}</a>" }
+            .join(' ')
+        end
+
+        if og_data[:video]
+          content_url = ::Onebox::Helpers.normalize_url_for_output(og_data[:video][:contentUrl])
+          video_url = Pathname.new(content_url)
+          @data[:webmUrl] = video_url.sub_ext(".webm").to_s
+          @data[:mp4Url] = video_url.sub_ext(".mp4").to_s
+
+          thumbnail_url = ::Onebox::Helpers.normalize_url_for_output(og_data[:video][:thumbnailUrl])
+          @data[:posterUrl] = thumbnail_url
+
+          @data[:width] = og_data[:video][:width]
+          @data[:height] = og_data[:video][:height]
+        end
+
+        @data
+      end
+    end
+  end
+end

lib/onebox/engine/giphy_onebox.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class GiphyOnebox
+      include Engine
+      include StandardEmbed
+
+      matches_regexp(/^https?:\/\/(giphy\.com\/gifs|gph\.is)\//)
+      always_https
+
+      def to_html
+        oembed = get_oembed
+
+        <<-HTML
+          <a href="#{oembed.url}" target="_blank" rel="noopener" class="onebox">
+            <img src="#{oembed.url}" width="#{oembed.width}" height="#{oembed.height}" #{oembed.title_attr}>
+          </a>
+        HTML
+      end
+    end
+  end
+end

lib/onebox/engine/github_blob_onebox.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+require_relative '../mixins/git_blob_onebox'
+
+module Onebox
+  module Engine
+    class GithubBlobOnebox
+      def self.git_regexp
+        /^https?:\/\/(www\.)?github\.com.*\/blob\//
+      end
+
+      def self.onebox_name
+        "githubblob"
+      end
+
+      include Onebox::Mixins::GitBlobOnebox
+
+      def raw_regexp
+        /github\.com\/(?<user>[^\/]+)\/(?<repo>[^\/]+)\/blob\/(?<sha1>[^\/]+)\/(?<file>[^#]+)(#(L(?<from>[^-]*)(-L(?<to>.*))?))?/mi
+      end
+
+      def raw_template(m)
+        "https://raw.githubusercontent.com/#{m[:user]}/#{m[:repo]}/#{m[:sha1]}/#{m[:file]}"
+      end
+
+      def title
+        Sanitize.fragment(Onebox::Helpers.uri_unencode(link).sub(/^https?\:\/\/github\.com\//, ''))
+      end
+    end
+  end
+end

lib/onebox/engine/github_commit_onebox.rb

@@ -0,0 +1,51 @@
+# frozen_string_literal: true
+
+require_relative '../mixins/github_body'
+
+module Onebox
+  module Engine
+    class GithubCommitOnebox
+      include Engine
+      include LayoutSupport
+      include JSON
+      include Onebox::Mixins::GithubBody
+
+      matches_regexp(/^https?:\/\/(?:www\.)?(?:(?:\w)+\.)?(github)\.com(?:\/)?(?:.)*\/commit\//)
+      always_https
+
+      def url
+        "https://api.github.com/repos/#{match[:owner]}/#{match[:repository]}/commits/#{match[:sha]}"
+      end
+
+      private
+
+      def match
+        return @match if defined?(@match)
+
+        @match = @url.match(%{github\.com/(?<owner>[^/]+)/(?<repository>[^/]+)/commit/(?<sha>[^/]+)})
+        @match ||= @url.match(%{github\.com/(?<owner>[^/]+)/(?<repository>[^/]+)/pull/(?<pr>[^/]+)/commit/(?<sha>[^/]+)})
+
+        @match
+      end
+
+      def data
+        result = raw.clone
+
+        lines = result['commit']['message'].split("\n")
+        result['title'] = lines.first
+        result['body'], result['excerpt'] = compute_body(lines[1..lines.length].join("\n"))
+
+        committed_at = Time.parse(result['commit']['author']['date'])
+        result['committed_at'] = committed_at.strftime("%I:%M%p - %d %b %y %Z")
+        result['committed_at_date'] = committed_at.strftime("%F")
+        result['committed_at_time'] = committed_at.strftime("%T")
+
+        result['link'] = link
+        ulink = URI(link)
+        result['domain'] = "#{ulink.host}/#{ulink.path.split('/')[1]}/#{ulink.path.split('/')[2]}"
+
+        result
+      end
+    end
+  end
+end

lib/onebox/engine/github_folder_onebox.rb

@@ -0,0 +1,78 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class GithubFolderOnebox
+      include Engine
+      include StandardEmbed
+      include LayoutSupport
+
+      matches_regexp(/^https?:\/\/(?:www\.)?(?:(?:\w)+\.)?(github)\.com[\:\d]*(\/[^\/]+){2}/)
+      always_https
+
+      def self.priority
+        # This engine should have lower priority than the other Github engines
+        150
+      end
+
+      private
+
+      def data
+        og = get_opengraph
+
+        max_length = 250
+
+        display_path = extract_path(og.url, max_length)
+        display_description = clean_description(og.description, og.title, max_length)
+
+        title = og.title
+
+        fragment = Addressable::URI.parse(url).fragment
+        if fragment
+          fragment = Addressable::URI.unencode(fragment)
+
+          if html_doc.css('.Box.md')
+            # For links to markdown docs
+            node = html_doc.css('a.anchor').find { |n| n['href'] == "##{fragment}" }
+            subtitle = node&.parent&.text
+          elsif html_doc.css('.Box.rdoc')
+            # For links to rdoc docs
+            node = html_doc.css('h3').find { |n| n['id'] == "user-content-#{fragment.downcase}" }
+            subtitle = node&.css('text()')&.first&.text
+          end
+
+          title = "#{title} - #{subtitle}" if subtitle
+        end
+
+        {
+          link: url,
+          image: og.image,
+          title: Onebox::Helpers.truncate(title, 250),
+          path: display_path,
+          description: display_description,
+          favicon: get_favicon
+        }
+      end
+
+      def extract_path(root, max_length)
+        path = url.split('#')[0].split('?')[0]
+        path = path["#{root}/tree/".length..-1]
+
+        return unless path
+
+        path.length > max_length ? path[-max_length..-1] : path
+      end
+
+      def clean_description(description, title, max_length)
+        return unless description
+
+        desc_end = " - #{title}"
+        if description[-desc_end.length..-1] == desc_end
+          description = description[0...-desc_end.length]
+        end
+
+        Onebox::Helpers.truncate(description, max_length)
+      end
+    end
+  end
+end

lib/onebox/engine/github_gist_onebox.rb

@@ -0,0 +1,81 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class GithubGistOnebox
+      include Engine
+      include LayoutSupport
+      include JSON
+
+      MAX_FILES = 3
+
+      matches_regexp(/^http(?:s)?:\/\/gist\.(?:(?:\w)+\.)?(github)\.com(?:\/)?/)
+      always_https
+
+      def url
+        "https://api.github.com/gists/#{match[:sha]}"
+      end
+
+      private
+
+      def data
+        @data ||= {
+          title: 'gist.github.com',
+          link: link,
+          gist_files: gist_files.take(MAX_FILES),
+          truncated_files?: truncated_files?
+        }
+      end
+
+      def truncated_files?
+        gist_files.size > MAX_FILES
+      end
+
+      def gist_files
+        return [] unless gist_api
+
+        @gist_files ||= gist_api["files"].values.map do |file_json|
+          GistFile.new(file_json)
+        end
+      end
+
+      def gist_api
+        @raw ||= raw.clone
+      rescue OpenURI::HTTPError
+        # The Gist API rate limit of 60 requests per hour was reached.
+        nil
+      end
+
+      def match
+        @match ||= @url.match(%r{gist\.github\.com/([^/]+/)?(?<sha>[0-9a-f]+)})
+      end
+
+      class GistFile
+        attr_reader :filename
+        attr_reader :language
+
+        MAX_LINES = 10
+
+        def initialize(json)
+          @json = json
+          @filename = @json["filename"]
+          @language = @json["language"]
+        end
+
+        def content
+          lines.take(MAX_LINES).join("\n")
+        end
+
+        def truncated?
+          lines.size > MAX_LINES
+        end
+
+        private
+
+        def lines
+          @lines ||= @json["content"].split("\n")
+        end
+      end
+    end
+  end
+end

lib/onebox/engine/github_issue_onebox.rb

@@ -0,0 +1,54 @@
+# frozen_string_literal: true
+
+require_relative '../mixins/github_body'
+
+module Onebox
+  module Engine
+    class GithubIssueOnebox
+      #Author Lidlanca 2014
+      include Engine
+      include LayoutSupport
+      include JSON
+      include Onebox::Mixins::GithubBody
+
+      matches_regexp(/^https?:\/\/(?:www\.)?(?:(?:\w)+\.)?github\.com\/(?<org>.+)\/(?<repo>.+)\/issues\/([[:digit:]]+)/)
+      always_https
+
+      def url
+        m = match
+        "https://api.github.com/repos/#{m["org"]}/#{m["repo"]}/issues/#{m["item_id"]}"
+      end
+
+      private
+
+      def match
+        @match ||= @url.match(/^http(?:s)?:\/\/(?:www\.)?(?:(?:\w)+\.)?github\.com\/(?<org>.+)\/(?<repo>.+)\/(?<type>issues)\/(?<item_id>[\d]+)/)
+      end
+
+      def data
+        created_at = Time.parse(raw['created_at'])
+        closed_at = Time.parse(raw['closed_at']) if raw['closed_at']
+        body, excerpt = compute_body(raw['body'])
+        ulink = URI(link)
+
+        {
+          link: @url,
+          title: raw["title"],
+          body: body,
+          excerpt: excerpt,
+          labels: raw["labels"],
+          user: raw['user'],
+          created_at: created_at.strftime("%I:%M%p - %d %b %y %Z"),
+          created_at_date: created_at.strftime("%F"),
+          created_at_time: created_at.strftime("%T"),
+          closed_at: closed_at&.strftime("%I:%M%p - %d %b %y %Z"),
+          closed_at_date: closed_at&.strftime("%F"),
+          closed_at_time: closed_at&.strftime("%T"),
+          closed_by: raw['closed_by'],
+          avatar: "https://avatars1.githubusercontent.com/u/#{raw['user']['id']}?v=2&s=96",
+          domain: "#{ulink.host}/#{ulink.path.split('/')[1]}/#{ulink.path.split('/')[2]}",
+        }
+      end
+    end
+  end
+end

lib/onebox/engine/github_pull_request_onebox.rb

@@ -0,0 +1,46 @@
+# frozen_string_literal: true
+
+require_relative '../mixins/github_body'
+
+module Onebox
+  module Engine
+    class GithubPullRequestOnebox
+      include Engine
+      include LayoutSupport
+      include JSON
+      include Onebox::Mixins::GithubBody
+
+      GITHUB_COMMENT_REGEX = /(<!--.*?-->\r\n)/
+
+      matches_regexp(/^https?:\/\/(?:www\.)?(?:(?:\w)+\.)?(github)\.com(?:\/)?(?:.)*\/pull/)
+      always_https
+
+      def url
+        "https://api.github.com/repos/#{match[:owner]}/#{match[:repository]}/pulls/#{match[:number]}"
+      end
+
+      private
+
+      def match
+        @match ||= @url.match(%r{github\.com/(?<owner>[^/]+)/(?<repository>[^/]+)/pull/(?<number>[^/]+)})
+      end
+
+      def data
+        result = raw.clone
+        result['link'] = link
+
+        created_at = Time.parse(result['created_at'])
+        result['created_at'] = created_at.strftime("%I:%M%p - %d %b %y %Z")
+        result['created_at_date'] = created_at.strftime("%F")
+        result['created_at_time'] = created_at.strftime("%T")
+
+        ulink = URI(link)
+        result['domain'] = "#{ulink.host}/#{ulink.path.split('/')[1]}/#{ulink.path.split('/')[2]}"
+
+        result['body'], result['excerpt'] = compute_body(result['body'])
+
+        result
+      end
+    end
+  end
+end

lib/onebox/engine/gitlab_blob_onebox.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+require_relative '../mixins/git_blob_onebox'
+
+module Onebox
+  module Engine
+    class GitlabBlobOnebox
+      def self.git_regexp
+        /^https?:\/\/(www\.)?gitlab\.com.*\/blob\//
+      end
+
+      def self.onebox_name
+        "gitlabblob"
+      end
+
+      include Onebox::Mixins::GitBlobOnebox
+
+      def raw_regexp
+        /gitlab\.com\/(?<user>[^\/]+)\/(?<repo>[^\/]+)\/blob\/(?<sha1>[^\/]+)\/(?<file>[^#]+)(#(L(?<from>[^-]*)(-L(?<to>.*))?))?/mi
+      end
+
+      def raw_template(m)
+        "https://gitlab.com/#{m[:user]}/#{m[:repo]}/raw/#{m[:sha1]}/#{m[:file]}"
+      end
+
+      def title
+        Sanitize.fragment(Onebox::Helpers.uri_unencode(link).sub(/^https?\:\/\/gitlab\.com\//, ''))
+      end
+    end
+  end
+end

lib/onebox/engine/google_calendar_onebox.rb

@@ -0,0 +1,32 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class GoogleCalendarOnebox
+      include Engine
+
+      matches_regexp(/^(https?:)?\/\/((www|calendar)\.google\.[\w.]{2,}|goo\.gl)\/calendar\/.+$/)
+      always_https
+      requires_iframe_origins "https://calendar.google.com"
+
+      def to_html
+        url = @url.split('&').first
+        src = ::Onebox::Helpers.normalize_url_for_output(url)
+        "<iframe src='#{src}&rm=minimal' style='border: 0' width='800' height='600' frameborder='0' scrolling='no'>#{placeholder_html}</iframe>"
+      end
+
+      def placeholder_html
+        <<-HTML
+          <div placeholder>
+            <div class='gdocs-onebox gdocs-onebox-splash' style='display:table-cell;vertical-align:middle;width:800px;height:600px'>
+              <div style='text-align:center;'>
+                <div class='gdocs-onebox-logo g-calendar-logo'></div>
+                <p>Google Calendar</p>
+              </div>
+            </div>
+          </div>
+        HTML
+      end
+    end
+  end
+end

lib/onebox/engine/google_docs_onebox.rb

@@ -0,0 +1,46 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class GoogleDocsOnebox
+      include Engine
+      include StandardEmbed
+      include LayoutSupport
+
+      SUPPORTED_ENDPOINTS = %w(spreadsheets document forms presentation)
+      SHORT_TYPES = {
+        spreadsheets: :sheets,
+        document: :docs,
+        presentation: :slides,
+        forms: :forms,
+      }
+
+      matches_regexp(/^(https?:)?\/\/(docs\.google\.com)\/(?<endpoint>(#{SUPPORTED_ENDPOINTS.join('|')}))\/d\/((?<key>[\w-]*)).+$/)
+      always_https
+
+      private
+
+      def data
+        og_data = get_opengraph
+        short_type = SHORT_TYPES[match[:endpoint].to_sym]
+
+        description = if Onebox::Helpers.blank?(og_data.description)
+          "This #{short_type.to_s.chop.capitalize} is private"
+        else
+          Onebox::Helpers.truncate(og_data.description, 250)
+        end
+
+        {
+          link: link,
+          title: og_data.title || "Google #{short_type.to_s.capitalize}",
+          description: description,
+          type: short_type
+        }
+      end
+
+      def match
+        @match ||= @url.match(@@matcher)
+      end
+    end
+  end
+end

lib/onebox/engine/google_drive_onebox.rb

@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class GoogleDriveOnebox
+      include Engine
+      include StandardEmbed
+      include LayoutSupport
+
+      matches_regexp(/^(https?:)?\/\/(drive\.google\.com)\/file\/d\/(?<key>[\w-]*)\/.+$/)
+      always_https
+
+      protected
+
+      def data
+        og_data = get_opengraph
+        title = og_data.title || "Google Drive"
+        title = "#{og_data.title} (video)" if og_data.type =~ /^video[\/\.]/
+        description = og_data.description || "Google Drive file."
+
+        {
+          link: link,
+          title: title,
+          description: Onebox::Helpers.truncate(description, 250),
+          image: og_data.image
+        }
+      end
+    end
+  end
+end

lib/onebox/engine/google_maps_onebox.rb

@@ -0,0 +1,184 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class GoogleMapsOnebox
+      include Engine
+
+      class << self
+        def ===(other)
+          if other.kind_of? URI
+            @@matchers && @@matchers.any? { |m| other.to_s =~ m[:regexp] }
+          else
+            super
+          end
+        end
+
+        private
+
+        def matches_regexp(key, regexp)
+          (@@matchers ||= []) << { key: key, regexp: regexp }
+        end
+      end
+
+      always_https
+      requires_iframe_origins("https://maps.google.com", "https://google.com")
+
+      # Matches shortened Google Maps URLs
+      matches_regexp :short,      %r"^(https?:)?//goo\.gl/maps/"
+
+      # Matches URLs for custom-created maps
+      matches_regexp :custom,     %r"^(?:https?:)?//www\.google(?:\.(?:\w{2,}))+/maps/d/(?:edit|viewer|embed)\?mid=.+$"
+
+      # Matches URLs with streetview data
+      matches_regexp :streetview, %r"^(?:https?:)?//www\.google(?:\.(?:\w{2,}))+/maps[^@]+@(?<lon>-?[\d.]+),(?<lat>-?[\d.]+),(?:\d+)a,(?<zoom>[\d.]+)y,(?<heading>[\d.]+)h,(?<pitch>[\d.]+)t.+?data=.*?!1s(?<pano>[^!]{22})"
+
+      # Matches "normal" Google Maps URLs with arbitrary data
+      matches_regexp :standard,   %r"^(?:https?:)?//www\.google(?:\.(?:\w{2,}))+/maps"
+
+      # Matches URLs for the old Google Maps domain which we occasionally get redirected to
+      matches_regexp :canonical,  %r"^(?:https?:)?//maps\.google(?:\.(?:\w{2,}))+/maps\?"
+
+      def initialize(url, timeout = nil)
+        super
+        resolve_url!
+      rescue Net::HTTPServerException, Timeout::Error, Net::HTTPError, Errno::ECONNREFUSED, RuntimeError => err
+        raise ArgumentError, "malformed url or unresolveable: #{err.message}"
+      end
+
+      def streetview?
+        !!@streetview
+      end
+
+      def to_html
+        "<div class='maps-onebox'><iframe src=\"#{link}\" width=\"690\" height=\"400\" frameborder=\"0\" style=\"border:0\">#{placeholder_html}</iframe></div>"
+      end
+
+      def placeholder_html
+        ::Onebox::Helpers.map_placeholder_html
+      end
+
+      private
+
+      def data
+        { link: url, title: url }
+      end
+
+      def resolve_url!
+        @streetview = false
+        type, match = match_url
+
+        # Resolve shortened URL, if necessary
+        if type == :short
+          follow_redirect!
+          type, match = match_url
+        end
+
+        # Try to get the old-maps URI, it is far easier to embed.
+        if type == :standard
+          retry_count = 10
+          while (retry_count -= 1) > 0
+            follow_redirect!
+            type, match = match_url
+            break if type != :standard
+            sleep 0.1
+          end
+        end
+
+        case type
+        when :standard
+          # Fallback for map URLs that don't resolve into an easily embeddable old-style URI
+          # Roadmaps use a "z" zoomlevel, satellite maps use "m" the horizontal width in meters
+          # TODO: tilted satellite maps using "a,y,t"
+          match = @url.match(/@(?<lon>[\d.-]+),(?<lat>[\d.-]+),(?<zoom>\d+)(?<mz>[mz])/)
+          raise "unexpected standard url #{@url}" unless match
+          zoom = match[:mz] == "z" ? match[:zoom] : Math.log2(57280048.0 / match[:zoom].to_f).round
+          location = "#{match[:lon]},#{match[:lat]}"
+          url = "https://maps.google.com/maps?ll=#{location}&z=#{zoom}&output=embed&dg=ntvb"
+          url += "&q=#{$1}" if match = @url.match(/\/place\/([^\/\?]+)/)
+          url += "&cid=#{($1 + $2).to_i(16)}" if @url.match(/!3m1!1s0x(\h{16}):0x(\h{16})/)
+          @url = url
+          @placeholder = "https://maps.googleapis.com/maps/api/staticmap?maptype=roadmap&center=#{location}&zoom=#{zoom}&size=690x400&sensor=false"
+
+        when :custom
+          url = @url.dup
+          @url = rewrite_custom_url(url, "embed")
+          @placeholder = rewrite_custom_url(url, "thumbnail")
+          @placeholder_height = @placeholder_width = 120
+
+        when :streetview
+          @streetview = true
+          panoid = match[:pano]
+          lon = match[:lon].to_f.to_s
+          lat = match[:lat].to_f.to_s
+          heading = match[:heading].to_f.round(4).to_s
+          pitch = (match[:pitch].to_f / 10.0).round(4).to_s
+          fov = (match[:zoom].to_f / 100.0).round(4).to_s
+          zoom = match[:zoom].to_f.round
+          @url = "https://www.google.com/maps/embed?pb=!3m2!2sen!4v0!6m8!1m7!1s#{panoid}!2m2!1d#{lon}!2d#{lat}!3f#{heading}!4f#{pitch}!5f#{fov}"
+          @placeholder = "https://maps.googleapis.com/maps/api/streetview?size=690x400&location=#{lon},#{lat}&pano=#{panoid}&fov=#{zoom}&heading=#{heading}&pitch=#{pitch}&sensor=false"
+
+        when :canonical
+          query = URI::decode_www_form(uri.query).to_h
+          if !query.has_key?("ll")
+            raise ArgumentError, "canonical url lacks location argument" unless query.has_key?("sll")
+            query["ll"] = query["sll"]
+            @url += "&ll=#{query["sll"]}"
+          end
+          location = query["ll"]
+          if !query.has_key?("z")
+            raise ArgumentError, "canonical url has incomplete query arguments" unless query.has_key?("spn") || query.has_key?("sspn")
+            if !query.has_key?("spn")
+              query["spn"] = query["sspn"]
+              @url += "&spn=#{query["sspn"]}"
+            end
+            angle = query["spn"].split(",").first.to_f
+            zoom = (Math.log(690.0 * 360.0 / angle / 256.0) / Math.log(2)).round
+          else
+            zoom = query["z"]
+          end
+          @url = @url.sub('output=classic', 'output=embed')
+          @placeholder = "https://maps.googleapis.com/maps/api/staticmap?maptype=roadmap&size=690x400&sensor=false&center=#{location}&zoom=#{zoom}"
+
+        else
+          raise "unexpected url type #{type.inspect}"
+        end
+      end
+
+      def match_url
+        @@matchers.each do |matcher|
+          if m = matcher[:regexp].match(@url)
+            return matcher[:key], m
+          end
+        end
+        raise ArgumentError, "\"#{@url}\" does not match any known pattern"
+      end
+
+      def rewrite_custom_url(url, target)
+        uri = URI(url)
+        uri.path = uri.path.sub(/(?<=^\/maps\/d\/)\w+$/, target)
+        uri.to_s
+      end
+
+      def follow_redirect!
+        begin
+          http = Net::HTTP.start(
+            uri.host,
+            uri.port,
+            use_ssl: uri.scheme == 'https',
+            open_timeout: timeout,
+            read_timeout: timeout
+          )
+
+          response = http.head(uri.path)
+          raise "unexpected response code #{response.code}" unless %w(200 301 302).include?(response.code)
+
+          @url = response.code == "200" ? uri.to_s : response["Location"]
+          @uri = URI(@url)
+        ensure
+          http.finish rescue nil
+        end
+      end
+    end
+  end
+end

lib/onebox/engine/google_photos_onebox.rb

@@ -0,0 +1,73 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class GooglePhotosOnebox
+      include Engine
+      include StandardEmbed
+
+      matches_regexp(/^https?:\/\/(photos)\.(app\.goo\.gl|google\.com)/)
+      always_https
+
+      def to_html
+        og = get_opengraph
+        return video_html(og) if og.video_secure_url
+        return album_html(og) if og.type == "google_photos:photo_album"
+        return image_html(og) if og.image
+        nil
+      end
+
+      private
+
+      def video_html(og)
+        escaped_url = ::Onebox::Helpers.normalize_url_for_output(url)
+
+        <<-HTML
+          <aside class="onebox google-photos">
+            <header class="source">
+              <img src="#{raw[:favicon]}" class="site-icon" width="16" height="16">
+              <a href="#{escaped_url}" target="_blank" rel="nofollow ugc noopener">#{raw[:site_name]}</a>
+            </header>
+            <article class="onebox-body">
+              <h3><a href="#{escaped_url}" target="_blank" rel="nofollow ugc noopener">#{og.title}</a></h3>
+              <div class="aspect-image-full-size">
+                <a href="#{escaped_url}" target="_blank" rel="nofollow ugc noopener">
+                  <img src="#{og.secure_image_url}" class="scale-image"/>
+                  <span class="instagram-video-icon"></span>
+                </a>
+              </div>
+            </article>
+          </aside>
+        HTML
+      end
+
+      def album_html(og)
+        escaped_url = ::Onebox::Helpers.normalize_url_for_output(url)
+        album_title = og.description.nil? ? og.title : "[#{og.description}] #{og.title}"
+
+        <<-HTML
+          <div class='onebox google-photos-album'>
+            <a href='#{escaped_url}' target='_blank' rel='noopener'>
+              <span class='outer-box' style='width:#{og.image_width}px'>
+                <span class='inner-box'>
+                  <span class='album-title'>#{Onebox::Helpers.truncate(album_title, 80)}</span>
+                </span>
+              </span>
+              <img src='#{og.secure_image_url}' #{og.title_attr} height='#{og.image_height}' width='#{og.image_width}'>
+            </a>
+          </div>
+        HTML
+      end
+
+      def image_html(og)
+        escaped_url = ::Onebox::Helpers.normalize_url_for_output(url)
+
+        <<-HTML
+          <a href='#{escaped_url}' target='_blank' rel='noopener' class="onebox">
+            <img src='#{og.secure_image_url}' #{og.title_attr} alt='Google Photos' height='#{og.image_height}' width='#{og.image_width}'>
+          </a>
+        HTML
+      end
+    end
+  end
+end

lib/onebox/engine/google_play_app_onebox.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class GooglePlayAppOnebox
+      include Engine
+      include LayoutSupport
+      include HTML
+
+      DEFAULTS = {
+        MAX_DESCRIPTION_CHARS: 500
+      }
+
+      matches_regexp(/^https?:\/\/play\.(?:(?:\w)+\.)?(google)\.com(?:\/)?\/store\/apps\//)
+      always_https
+
+      private
+
+      def data
+        price = raw.css("meta[itemprop=price]").first["content"] rescue "Free"
+        {
+          link: link,
+          title: raw.css("meta[property='og:title']").first["content"].gsub(" - Apps on Google Play", ""),
+          image: ::Onebox::Helpers.normalize_url_for_output(raw.css("meta[property='og:image']").first["content"]),
+          description: raw.css("meta[name=description]").first["content"][0..DEFAULTS[:MAX_DESCRIPTION_CHARS]].chop + "...",
+          price: price == "0" ? "Free" : price
+        }
+      end
+    end
+  end
+end

lib/onebox/engine/html.rb

@@ -0,0 +1,26 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    module HTML
+      private
+
+      # Overwrite for any custom headers
+      def http_params
+        {}
+      end
+
+      def raw
+        @raw ||= Onebox::Helpers.fetch_html_doc(url, http_params, body_cacher)
+      end
+
+      def body_cacher
+        self.options&.[](:body_cacher)
+      end
+
+      def html?
+        raw.respond_to(:css)
+      end
+    end
+  end
+end

lib/onebox/engine/image_onebox.rb

@@ -0,0 +1,29 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class ImageOnebox
+      include Engine
+
+      matches_regexp(/^(https?:)?\/\/.+\.(png|jpg|jpeg|gif|bmp|tif|tiff)(\?.*)?$/i)
+
+      def always_https?
+        AllowlistedGenericOnebox.host_matches(uri, AllowlistedGenericOnebox.https_hosts)
+      end
+
+      def to_html
+        # Fix Dropbox image links
+        if @url[/^https:\/\/www.dropbox.com\/s\//]
+          @url.sub!("https://www.dropbox.com", "https://dl.dropboxusercontent.com")
+        end
+
+        escaped_url = ::Onebox::Helpers.normalize_url_for_output(@url)
+        <<-HTML
+          <a href="#{escaped_url}" target="_blank" rel="noopener" class="onebox">
+            <img src="#{escaped_url}">
+          </a>
+        HTML
+      end
+    end
+  end
+end

lib/onebox/engine/imgur_onebox.rb

@@ -0,0 +1,67 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class ImgurOnebox
+      include Engine
+      include StandardEmbed
+
+      matches_regexp(/^https?:\/\/(www\.)?imgur\.com/)
+      always_https
+
+      def to_html
+        og = get_opengraph
+        return video_html(og) if !og.video_secure_url.nil?
+        return album_html(og) if is_album?
+        return image_html(og) if !og.image.nil?
+        nil
+      end
+
+      private
+
+      def video_html(og)
+        <<-HTML
+            <video width='#{og.video_width}' height='#{og.video_height}' #{og.title_attr} controls loop>
+              <source src='#{og.video_secure_url}' type='video/mp4'>
+              <source src='#{og.video_secure_url.gsub('mp4', 'webm')}' type='video/webm'>
+            </video>
+          HTML
+      end
+
+      def album_html(og)
+        escaped_url = ::Onebox::Helpers.normalize_url_for_output(url)
+        album_title = "[Album] #{og.title}"
+
+        <<-HTML
+            <div class='onebox imgur-album'>
+              <a href='#{escaped_url}' target='_blank' rel='noopener'>
+                <span class='outer-box' style='width:#{og.image_width}px'>
+                  <span class='inner-box'>
+                    <span class='album-title'>#{album_title}</span>
+                  </span>
+                </span>
+                <img src='#{og.secure_image_url}' #{og.title_attr} height='#{og.image_height}' width='#{og.image_width}'>
+              </a>
+            </div>
+          HTML
+      end
+
+      def is_album?
+        response = Onebox::Helpers.fetch_response("https://api.imgur.com/oembed.json?url=#{url}") rescue "{}"
+        oembed_data = Onebox::Helpers.symbolize_keys(::MultiJson.load(response))
+        imgur_data_id = Nokogiri::HTML(oembed_data[:html]).xpath("//blockquote").attr("data-id")
+        imgur_data_id.to_s[/a\//]
+      end
+
+      def image_html(og)
+        escaped_url = ::Onebox::Helpers.normalize_url_for_output(url)
+
+        <<-HTML
+            <a href='#{escaped_url}' target='_blank' rel='noopener' class="onebox">
+              <img src='#{og.secure_image_url.chomp("?fb")}' #{og.title_attr} alt='Imgur'>
+            </a>
+          HTML
+      end
+    end
+  end
+end

lib/onebox/engine/instagram_onebox.rb

@@ -0,0 +1,46 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class InstagramOnebox
+      include Engine
+      include StandardEmbed
+      include LayoutSupport
+
+      matches_regexp(/^https?:\/\/(?:www\.)?(?:instagram\.com|instagr\.am)\/?(?:.*)\/(?:p|tv)\/[a-zA-Z\d_-]+/)
+      always_https
+
+      def clean_url
+        url.scan(/^https?:\/\/(?:www\.)?(?:instagram\.com|instagr\.am)\/?(?:.*)\/(?:p|tv)\/[a-zA-Z\d_-]+/).flatten.first
+      end
+
+      def data
+        oembed = get_oembed
+        raise "No oEmbed data found. Ensure 'facebook_app_access_token' is valid" if oembed.data.empty?
+
+        {
+          link: clean_url.gsub("/#{oembed.author_name}/", "/"),
+          title: "@#{oembed.author_name}",
+          image: oembed.thumbnail_url,
+          description: Onebox::Helpers.truncate(oembed.title, 250),
+        }
+
+      end
+
+      protected
+
+      def access_token
+        (options[:facebook_app_access_token] || Onebox.options.facebook_app_access_token).to_s
+      end
+
+      def get_oembed_url
+        if access_token != ''
+          "https://graph.facebook.com/v9.0/instagram_oembed?url=#{clean_url}&access_token=#{access_token}"
+        else
+          # The following is officially deprecated by Instagram, but works in some limited circumstances.
+          "https://api.instagram.com/oembed/?url=#{clean_url}"
+        end
+      end
+    end
+  end
+end

lib/onebox/engine/json.rb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    module JSON
+      private
+
+      def raw
+        @raw ||= ::MultiJson.load(URI.open(url, read_timeout: timeout))
+      end
+    end
+  end
+end

lib/onebox/engine/kaltura_onebox.rb

@@ -0,0 +1,36 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class KalturaOnebox
+      include Engine
+      include StandardEmbed
+
+      always_https
+      matches_regexp(/^https?:\/\/[a-z0-9]+\.kaltura\.com\/id\/[a-zA-Z0-9]+/)
+      requires_iframe_origins "https://*.kaltura.com"
+
+      def preview_html
+        og = get_opengraph
+
+        <<~HTML
+          <img src="#{og.image_secure_url}" width="#{og.video_width}" height="#{og.video_height}">
+        HTML
+      end
+
+      def to_html
+        og = get_opengraph
+
+        <<~HTML
+          <iframe
+            src="#{og.video_secure_url}"
+            width="#{og.video_width}"
+            height="#{og.video_height}"
+            frameborder='0'
+            allowfullscreen
+          ></iframe>
+        HTML
+      end
+    end
+  end
+end

lib/onebox/engine/mixcloud_onebox.rb

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class MixcloudOnebox
+      include Engine
+      include StandardEmbed
+
+      matches_regexp(/^https?:\/\/www\.mixcloud\.com\//)
+      always_https
+
+      def placeholder_html
+        oembed = get_oembed
+        "<img src='#{oembed.image}' height='#{oembed.height}' #{oembed.title_attr}>"
+      end
+
+      def to_html
+        get_oembed.html
+      end
+    end
+  end
+end

lib/onebox/engine/opengraph_image.rb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    module OpengraphImage
+
+      def to_html
+        og = get_opengraph
+        "<img src='#{og.image}' width='#{og.image_width}' height='#{og.image_height}' class='onebox' #{og.title_attr}>"
+      end
+    end
+  end
+end

lib/onebox/engine/pastebin_onebox.rb

@@ -0,0 +1,55 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class PastebinOnebox
+      include Engine
+      include LayoutSupport
+
+      MAX_LINES = 10
+
+      matches_regexp(/^http?:\/\/pastebin\.com/)
+
+      private
+
+      def data
+        @data ||= {
+          title: 'pastebin.com',
+          link: link,
+          content: content,
+          truncated?: truncated?
+        }
+      end
+
+      def content
+        lines.take(MAX_LINES).join("\n")
+      end
+
+      def truncated?
+        lines.size > MAX_LINES
+      end
+
+      def lines
+        return @lines if defined?(@lines)
+        response = Onebox::Helpers.fetch_response("http://pastebin.com/raw/#{paste_key}", redirect_limit: 1) rescue ""
+        @lines = response.split("\n")
+      end
+
+      def paste_key
+        regex = case uri
+                when /\/raw\//
+                  /\/raw\/([^\/]+)/
+                when /\/download\//
+                  /\/download\/([^\/]+)/
+                when /\/embed\//
+                  /\/embed\/([^\/]+)/
+        else
+                  /\/([^\/]+)/
+        end
+
+        match = uri.path.match(regex)
+        match[1] if match && match[1]
+      end
+    end
+  end
+end

lib/onebox/engine/pdf_onebox.rb

@@ -0,0 +1,29 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class PdfOnebox
+      include Engine
+      include LayoutSupport
+
+      matches_regexp(/^(https?:)?\/\/.*\.pdf(\?.*)?$/i)
+      always_https
+
+      private
+
+      def data
+        begin
+          size = Onebox::Helpers.fetch_content_length(@url)
+        rescue
+          raise "Unable to read pdf file: #{@url}"
+        end
+
+        {
+          link: link,
+          title: File.basename(uri.path),
+          filesize: size ? Onebox::Helpers.pretty_filesize(size.to_i) : nil,
+        }
+      end
+    end
+  end
+end

lib/onebox/engine/pubmed_onebox.rb

@@ -0,0 +1,60 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class PubmedOnebox
+      include Engine
+      include LayoutSupport
+
+      matches_regexp(/^https?:\/\/(?:(?:\w)+\.)?(www.ncbi.nlm.nih)\.gov(?:\/)?\/pubmed\/\d+/)
+
+      private
+
+      def xml
+        return @xml if defined?(@xml)
+        doc = Nokogiri::XML(URI.open(URI.join(@url, "?report=xml&format=text")))
+        pre = doc.xpath("//pre")
+        @xml = Nokogiri::XML("<root>" + pre.text + "</root>")
+      end
+
+      def authors
+        initials = xml.css("Initials").map { |x| x.content }
+        last_names = xml.css("LastName").map { |x| x.content }
+        author_list = (initials.zip(last_names)).map { |i, l| i + " " + l }
+        if author_list.length > 1 then
+          author_list[-2] = author_list[-2] + " and " + author_list[-1]
+          author_list.pop
+        end
+        author_list.join(", ")
+      end
+
+      def date
+        xml.css("PubDate")
+          .children
+          .map { |x| x.content }
+          .select { |s| !s.match(/^\s+$/) }
+          .map { |s| s.split }
+          .flatten
+          .sort
+          .reverse
+          .join(" ") # Reverse sort so month before year.
+      end
+
+      def data
+        {
+          title: xml.css("ArticleTitle").text,
+          authors: authors,
+          journal: xml.css("Title").text,
+          abstract: xml.css("AbstractText").text,
+          date: date,
+          link: @url,
+          pmid: match[:pmid]
+        }
+      end
+
+      def match
+        @match ||= @url.match(%r{www\.ncbi\.nlm\.nih\.gov/pubmed/(?<pmid>[0-9]+)})
+      end
+    end
+  end
+end

lib/onebox/engine/reddit_media_onebox.rb

@@ -0,0 +1,55 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class RedditMediaOnebox
+      include Engine
+      include StandardEmbed
+
+      matches_regexp(/^https?:\/\/(www\.)?reddit\.com/)
+
+      def to_html
+        if raw[:type] == "image"
+          <<-HTML
+            <aside class="onebox reddit">
+              <header class="source">
+                <img src="#{raw[:favicon]}" class="site-icon" width="16" height="16">
+                <a href="#{raw[:url]}" target="_blank" rel="nofollow ugc noopener">#{raw[:site_name]}</a>
+              </header>
+              <article class="onebox-body">
+                <h3><a href="#{raw[:url]}" target="_blank" rel="nofollow ugc noopener">#{raw[:title]}</a></h3>
+                <div class="scale-images">
+                  <img src="#{raw[:image]}" class="scale-image"/>
+                </div>
+                <div class="description"><p>#{raw[:description]}</p></div>
+              </article>
+            </aside>
+          HTML
+        elsif  raw[:type] =~ /^video[\/\.]/
+          <<-HTML
+            <aside class="onebox reddit">
+              <header class="source">
+                <img src="#{raw[:favicon]}" class="site-icon" width="16" height="16">
+                <a href="#{raw[:url]}" target="_blank" rel="nofollow ugc noopener">#{raw[:site_name]}</a>
+              </header>
+              <article class="onebox-body">
+                <h3><a href="#{raw[:url]}" target="_blank" rel="nofollow ugc noopener">#{raw[:title]}</a></h3>
+                <div class="aspect-image-full-size">
+                  <a href="#{raw[:url]}" target="_blank" rel="nofollow ugc noopener">
+                    <img src="#{raw[:image]}" class="scale-image"/>
+                    <span class="instagram-video-icon"></span>
+                  </a>
+                </div>
+                <div class="description"><p>#{raw[:description]}</p></div>
+              </article>
+            </aside>
+          HTML
+        else
+          html = Onebox::Engine::AllowlistedGenericOnebox.new(@url, @timeout).to_html
+          return if Onebox::Helpers.blank?(html)
+          html
+        end
+      end
+    end
+  end
+end

lib/onebox/engine/replit_onebox.rb

@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class ReplitOnebox
+      include Engine
+      include StandardEmbed
+
+      matches_regexp(/^https?:\/\/repl\.it\/.+/)
+      always_https
+
+      def placeholder_html
+        oembed = get_oembed
+
+        <<-HTML
+          <img src="#{oembed.thumbnail_url}" style="max-width: #{oembed.width}px; max-height: #{oembed.height}px;" #{oembed.title_attr}>
+        HTML
+      end
+
+      def to_html
+        get_oembed.html
+      end
+    end
+  end
+end

lib/onebox/engine/simplecast_onebox.rb

@@ -0,0 +1,36 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class SimplecastOnebox
+      include Engine
+      include StandardEmbed
+
+      matches_regexp(/https?:\/\/(.+)?simplecast.com\/(episodes|s)\/.*/)
+      always_https
+      requires_iframe_origins("https://embed.simplecast.com")
+
+      def to_html
+        get_oembed.html
+      end
+
+      def placeholder_html
+        oembed = get_oembed
+        return if Onebox::Helpers.blank?(oembed.thumbnail_url)
+        "<img src='#{oembed.thumbnail_url}' #{oembed.title_attr}>"
+      end
+
+      private
+
+      def get_oembed_url
+        if id = url.scan(/([a-zA-Z0-9]*)\Z/).flatten.first
+          oembed_url = "https://simplecast.com/s/#{id}"
+        else
+          oembed_url = url
+        end
+
+        "https://simplecast.com/oembed?url=#{oembed_url}"
+      end
+    end
+  end
+end

lib/onebox/engine/sketch_fab_onebox.rb

@@ -0,0 +1,34 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class SketchFabOnebox
+      include Engine
+      include StandardEmbed
+
+      matches_regexp(/^https?:\/\/sketchfab\.com\/(?:models\/|3d-models\/(?:[^\/\s]+-)?)([a-z0-9]{32})/)
+      always_https
+      requires_iframe_origins("https://sketchfab.com")
+
+      def to_html
+        og = get_opengraph
+        src = og.video_url.gsub("autostart=1", "")
+
+        <<-HTML
+          <iframe
+            src="#{src}"
+            width="#{og.video_width}"
+            height="#{og.video_height}"
+            scrolling="no"
+            frameborder="0"
+            allowfullscreen
+          ></iframe>
+        HTML
+      end
+
+      def placeholder_html
+        "<img src='#{get_opengraph.image}'>"
+      end
+    end
+  end
+end

lib/onebox/engine/slides_onebox.rb

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class SlidesOnebox
+      include Engine
+      include StandardEmbed
+
+      matches_regexp(/^https?:\/\/slides\.com\/[\p{Alnum}_\-]+\/[\p{Alnum}_\-]+$/)
+      requires_iframe_origins "https://slides.com"
+
+      def to_html
+        <<-HTML
+          <iframe
+            src="https://slides.com#{uri.path}/embed?style=light"
+            width="576"
+            height="420"
+            scrolling="no"
+            frameborder="0"
+            webkitallowfullscreen
+            mozallowfullscreen
+            allowfullscreen
+          ></iframe>
+        HTML
+      end
+
+      def placeholder_html
+        escaped_src = ::Onebox::Helpers.normalize_url_for_output(raw[:image])
+        "<img src='#{escaped_src}'>"
+      end
+    end
+  end
+end

lib/onebox/engine/sound_cloud_onebox.rb

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class SoundCloudOnebox
+      include Engine
+      include StandardEmbed
+
+      matches_regexp(/^https?:\/\/soundcloud\.com/)
+      requires_iframe_origins "https://w.soundcloud.com"
+      always_https
+
+      def to_html
+        oembed = get_oembed
+        oembed.html.gsub('visual=true', 'visual=false')
+      end
+
+      def placeholder_html
+        oembed = get_oembed
+        return if Onebox::Helpers.blank?(oembed.thumbnail_url)
+        "<img src='#{oembed.thumbnail_url}' #{oembed.title_attr}>"
+      end
+
+      protected
+
+      def get_oembed_url
+        oembed_url = "https://soundcloud.com/oembed.json?url=#{url}"
+        oembed_url += "&maxheight=166" unless url["/sets/"]
+        oembed_url
+      end
+    end
+  end
+end

lib/onebox/engine/stack_exchange_onebox.rb

@@ -0,0 +1,56 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class StackExchangeOnebox
+      include Engine
+      include LayoutSupport
+      include JSON
+
+      def self.domains
+        %w(stackexchange.com stackoverflow.com superuser.com serverfault.com askubuntu.com stackapps.com mathoverflow.net)
+          .map { |domain| Regexp.escape(domain) }
+      end
+
+      matches_regexp(/^https?:\/\/(?:(?:(?<subsubdomain>\w*)\.)?(?<subdomain>\w*)\.)?(?<domain>#{domains.join('|')})\/((?:questions|q)\/(?<question_id>\d*)(\/.*\/(?<answer_id1>\d*))?|(a\/(?<answer_id2>\d*)))/)
+
+      def always_https?
+        uri.host.split('.').length <= 3
+      end
+
+      private
+
+      def match
+        @match ||= @url.match(@@matcher)
+      end
+
+      def url
+        domain = uri.host
+        question_id = match[:question_id]
+        answer_id = match[:answer_id2] || match[:answer_id1]
+
+        if answer_id
+          "https://api.stackexchange.com/2.2/answers/#{answer_id}?site=#{domain}&filter=!.FjueITQdx6-Rq3Ue9PWG.QZ2WNdW"
+        else
+          "https://api.stackexchange.com/2.2/questions/#{question_id}?site=#{domain}&filter=!5-duuxrJa-iw9oVvOA(JNimB5VIisYwZgwcfNI"
+        end
+      end
+
+      def data
+        return @data if defined?(@data)
+
+        result = raw['items'][0]
+        if result
+          result['creation_date'] =
+              Time.at(result['creation_date'].to_i).strftime("%I:%M%p - %d %b %y %Z")
+
+          result['tags'] = result['tags'].take(4).join(', ')
+          result['is_answer'] = result.key?('answer_id')
+          result['is_question'] = result.key?('question_id')
+        end
+
+        @data = result
+      end
+    end
+  end
+end

lib/onebox/engine/standard_embed.rb

@@ -0,0 +1,145 @@
+# frozen_string_literal: true
+
+require "cgi"
+require "onebox/open_graph"
+require 'onebox/oembed'
+
+module Onebox
+  module Engine
+    module StandardEmbed
+      def self.oembed_providers
+        @@oembed_providers ||= {}
+      end
+
+      def self.add_oembed_provider(regexp, endpoint)
+        oembed_providers[regexp] = endpoint
+      end
+
+      def self.opengraph_providers
+        @@opengraph_providers ||= []
+      end
+
+      def self.add_opengraph_provider(regexp)
+        opengraph_providers << regexp
+      end
+
+      # Some oembed providers (like meetup.com) don't provide links to themselves
+      add_oembed_provider(/www\.meetup\.com\//, 'http://api.meetup.com/oembed')
+      add_oembed_provider(/www\.mixcloud\.com\//, 'https://www.mixcloud.com/oembed/')
+      # In order to support Private Videos
+      add_oembed_provider(/vimeo\.com\//, 'https://vimeo.com/api/oembed.json')
+      # NYT requires login so use oembed only
+      add_oembed_provider(/nytimes\.com\//, 'https://www.nytimes.com/svc/oembed/json/')
+
+      def always_https?
+        AllowlistedGenericOnebox.host_matches(uri, AllowlistedGenericOnebox.https_hosts) || super
+      end
+
+      def raw
+        return @raw if defined?(@raw)
+
+        og = get_opengraph
+        twitter = get_twitter
+        oembed = get_oembed
+
+        @raw = {}
+
+        og.data.each do |k, v|
+          next if k == "title_attr"
+          v = og.send(k)
+          @raw[k] ||= v unless v.nil?
+        end
+
+        twitter.each { |k, v| @raw[k] ||= v unless Onebox::Helpers::blank?(v) }
+
+        oembed.data.each do |k, v|
+          v = oembed.send(k)
+          @raw[k] ||= v unless v.nil?
+        end
+
+        favicon = get_favicon
+        @raw["favicon".to_sym] = favicon unless Onebox::Helpers::blank?(favicon)
+
+        @raw
+      end
+
+      protected
+
+      def html_doc
+        return @html_doc if defined?(@html_doc)
+
+        headers = nil
+        headers = { 'Cookie' => options[:cookie] } if options[:cookie]
+
+        @html_doc = Onebox::Helpers.fetch_html_doc(url, headers)
+      end
+
+      def get_oembed
+        @oembed ||= Onebox::Oembed.new(get_json_response)
+      end
+
+      def get_opengraph
+        @opengraph ||= ::Onebox::OpenGraph.new(html_doc)
+      end
+
+      def get_twitter
+        return {} unless html_doc
+
+        twitter = {}
+
+        html_doc.css('meta').each do |m|
+          if (m["property"] && m["property"][/^twitter:(.+)$/i]) || (m["name"] && m["name"][/^twitter:(.+)$/i])
+            value = (m["content"] || m["value"]).to_s
+            twitter[$1.tr('-:' , '_').to_sym] ||= value unless (Onebox::Helpers::blank?(value) || value == "0 minutes")
+          end
+        end
+
+        twitter
+      end
+
+      def get_favicon
+        return nil unless html_doc
+
+        favicon = html_doc.css('link[rel="shortcut icon"], link[rel="icon shortcut"], link[rel="shortcut"], link[rel="icon"]').first
+        favicon = favicon.nil? ? nil : (favicon['href'].nil? ? nil : favicon['href'].strip)
+
+        Onebox::Helpers::get_absolute_image_url(favicon, url)
+      end
+
+      def get_json_response
+        oembed_url = get_oembed_url
+
+        return "{}" if Onebox::Helpers.blank?(oembed_url)
+
+        Onebox::Helpers.fetch_response(oembed_url) rescue "{}"
+      rescue Errno::ECONNREFUSED, Net::HTTPError, Net::HTTPFatalError, MultiJson::LoadError
+        "{}"
+      end
+
+      def get_oembed_url
+        oembed_url = nil
+
+        StandardEmbed.oembed_providers.each do |regexp, endpoint|
+          if url =~ regexp
+            oembed_url = "#{endpoint}?url=#{url}"
+            break
+          end
+        end
+
+        if html_doc
+          if Onebox::Helpers.blank?(oembed_url)
+            application_json = html_doc.at("//link[@type='application/json+oembed']/@href")
+            oembed_url = application_json.value if application_json
+          end
+
+          if Onebox::Helpers.blank?(oembed_url)
+            text_json = html_doc.at("//link[@type='text/json+oembed']/@href")
+            oembed_url ||= text_json.value if text_json
+          end
+        end
+
+        oembed_url
+      end
+    end
+  end
+end

lib/onebox/engine/steam_store_onebox.rb

@@ -0,0 +1,41 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class SteamStoreOnebox
+      include Engine
+      include StandardEmbed
+
+      always_https
+      matches_regexp(/^https?:\/\/store\.steampowered\.com\/app\/\d+/)
+      requires_iframe_origins "https://store.steampowered.com"
+
+      def placeholder_html
+        og = get_opengraph
+        <<-HTML
+          <div style='width:100%; height:190px; background-color:#262626; color:#9e9e9e; margin:15px 0;'>
+            <div style='padding:10px'>
+              <h3 style='color:#fff; margin:10px 0 10px 5px;'>#{og.title}</h3>
+              <img src='#{og.image}' style='float:left; max-width:184px; margin:5px 15px 0 5px'/>
+              <p>#{og.description}</p>
+            </div>
+          </div>
+        HTML
+      end
+
+      def to_html
+        iframe_url = @url[/https?:\/\/store\.steampowered\.com\/app\/\d+/].gsub("/app/", "/widget/")
+        escaped_src = ::Onebox::Helpers.normalize_url_for_output(iframe_url)
+
+        <<-HTML
+          <iframe
+            src='#{escaped_src}'
+            frameborder='0'
+            width='100%'
+            height='190'
+          ></iframe>
+        HTML
+      end
+    end
+  end
+end

lib/onebox/engine/trello_onebox.rb

@@ -0,0 +1,34 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class TrelloOnebox
+      include Engine
+      include StandardEmbed
+
+      matches_regexp(/^https:\/\/trello\.com\/[bc]\/\W*/)
+      requires_iframe_origins "https://trello.com"
+      always_https
+
+      def to_html
+        src = "https://trello.com/#{match[:type]}/#{match[:key]}.html"
+        height = match[:type] == 'b' ? 400 : 200
+
+        <<-HTML
+          <iframe src="#{src}" width="100%" height="#{height}" frameborder="0" style="border:0"></iframe>
+        HTML
+      end
+
+      def placeholder_html
+        ::Onebox::Helpers.generic_placeholder_html
+      end
+
+      private
+
+      def match
+        return @match if defined?(@match)
+        @match = @url.match(%{trello\.com/(?<type>[^/]+)/(?<key>[^/]+)/?\W*})
+      end
+    end
+  end
+end

lib/onebox/engine/twitch_clips_onebox.rb

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+
+require_relative '../mixins/twitch_onebox'
+
+class Onebox::Engine::TwitchClipsOnebox
+  def self.twitch_regexp
+    /^https?:\/\/clips\.twitch\.tv\/([a-zA-Z0-9_]+\/?[^#\?\/]+)/
+  end
+
+  include Onebox::Mixins::TwitchOnebox
+  requires_iframe_origins "https://clips.twitch.tv"
+
+  def query_params
+    "clip=#{twitch_id}"
+  end
+
+  def base_url
+    "clips.twitch.tv/embed?"
+  end
+end

lib/onebox/engine/twitch_stream_onebox.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+require_relative '../mixins/twitch_onebox'
+
+class Onebox::Engine::TwitchStreamOnebox
+  def self.twitch_regexp
+    /^https?:\/\/(?:www\.|go\.)?twitch\.tv\/(?!directory)([a-zA-Z0-9_]{4,25})$/
+  end
+
+  include Onebox::Mixins::TwitchOnebox
+
+  def query_params
+    "channel=#{twitch_id}"
+  end
+end

lib/onebox/engine/twitch_video_onebox.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+require_relative '../mixins/twitch_onebox'
+
+class Onebox::Engine::TwitchVideoOnebox
+  def self.twitch_regexp
+    /^https?:\/\/(?:www\.)?twitch\.tv\/videos\/([0-9]+)/
+  end
+
+  include Onebox::Mixins::TwitchOnebox
+
+  def query_params
+    "video=v#{twitch_id}"
+  end
+end

lib/onebox/engine/twitter_status_onebox.rb

@@ -0,0 +1,172 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class TwitterStatusOnebox
+      include Engine
+      include LayoutSupport
+      include HTML
+
+      matches_regexp(/^https?:\/\/(mobile\.|www\.)?twitter\.com\/.+?\/status(es)?\/\d+(\/(video|photo)\/\d?+)?+(\/?\?.*)?\/?$/)
+      always_https
+
+      def http_params
+        { 'User-Agent' => 'DiscourseBot/1.0' }
+      end
+
+      private
+
+      def get_twitter_data
+        response = Onebox::Helpers.fetch_response(url, headers: http_params) rescue nil
+        html = Nokogiri::HTML(response)
+        twitter_data = {}
+        html.css('meta').each do |m|
+          if m.attribute('property') && m.attribute('property').to_s.match(/^og:/i)
+            m_content = m.attribute('content').to_s.strip
+            m_property = m.attribute('property').to_s.gsub('og:', '')
+            twitter_data[m_property.to_sym] = m_content
+          end
+        end
+        twitter_data
+      end
+
+      def match
+        @match ||= @url.match(%r{twitter\.com/.+?/status(es)?/(?<id>\d+)})
+      end
+
+      def twitter_data
+        @twitter_data ||= get_twitter_data
+      end
+
+      def client
+        Onebox.options.twitter_client
+      end
+
+      def twitter_api_credentials_present?
+        client && !client.twitter_credentials_missing?
+      end
+
+      def raw
+        if twitter_api_credentials_present?
+          @raw ||= OpenStruct.new(client.status(match[:id]).to_hash)
+        else
+          super
+        end
+      end
+
+      def access(*keys)
+        keys.reduce(raw) do |memo, key|
+          next unless memo
+          memo[key] || memo[key.to_s]
+        end
+      end
+
+      def tweet
+        if twitter_api_credentials_present?
+          client.prettify_tweet(raw)&.strip
+        else
+          twitter_data[:description].gsub(/“(.+?)”/im) { $1 } if twitter_data[:description]
+        end
+      end
+
+      def timestamp
+        if twitter_api_credentials_present?
+          date = DateTime.strptime(access(:created_at), "%a %b %d %H:%M:%S %z %Y")
+          user_offset = access(:user, :utc_offset).to_i
+          offset = (user_offset >= 0 ? "+" : "-") + Time.at(user_offset.abs).gmtime.strftime("%H%M")
+          date.new_offset(offset).strftime("%-l:%M %p - %-d %b %Y")
+        else
+          attr_at_css(".tweet-timestamp", 'title')
+        end
+      end
+
+      def title
+        if twitter_api_credentials_present?
+          "#{access(:user, :name)} (#{access(:user, :screen_name)})"
+        else
+          "#{attr_at_css('.tweet.permalink-tweet', 'data-name')} (#{attr_at_css('.tweet.permalink-tweet', 'data-screen-name')})"
+        end
+      end
+
+      def avatar
+        if twitter_api_credentials_present?
+          access(:user, :profile_image_url_https).sub('normal', '400x400')
+        elsif twitter_data[:image]
+          twitter_data[:image]
+        end
+      end
+
+      def likes
+        if twitter_api_credentials_present?
+          prettify_number(access(:favorite_count).to_i)
+        else
+          attr_at_css(".request-favorited-popup", 'data-compact-localized-count')
+        end
+      end
+
+      def retweets
+        if twitter_api_credentials_present?
+          prettify_number(access(:retweet_count).to_i)
+        else
+          attr_at_css(".request-retweeted-popup", 'data-compact-localized-count')
+        end
+      end
+
+      def quoted_full_name
+        if twitter_api_credentials_present?
+          access(:quoted_status, :user, :name)
+        else
+          raw.css('.QuoteTweet-fullname')[0]&.text
+        end
+      end
+
+      def quoted_screen_name
+        if twitter_api_credentials_present?
+          access(:quoted_status, :user, :screen_name)
+        else
+          attr_at_css(".QuoteTweet-innerContainer", "data-screen-name")
+        end
+      end
+
+      def quoted_tweet
+        if twitter_api_credentials_present?
+          access(:quoted_status, :full_text)
+        else
+          raw.css('.QuoteTweet-text')[0]&.text
+        end
+      end
+
+      def quoted_link
+        if twitter_api_credentials_present?
+          "https://twitter.com/#{quoted_screen_name}/status/#{access(:quoted_status, :id)}"
+        else
+          "https://twitter.com#{attr_at_css(".QuoteTweet-innerContainer", "href")}"
+        end
+      end
+
+      def prettify_number(count)
+        count > 0 ? client.prettify_number(count) : nil
+      end
+
+      def attr_at_css(css_property, attribute_name)
+        raw.at_css(css_property)&.attr(attribute_name)
+      end
+
+      def data
+        @data ||= {
+          link: link,
+          tweet: tweet,
+          timestamp: timestamp,
+          title: title,
+          avatar: avatar,
+          likes: likes,
+          retweets: retweets,
+          quoted_tweet: quoted_tweet,
+          quoted_full_name: quoted_full_name,
+          quoted_screen_name: quoted_screen_name,
+          quoted_link: quoted_link
+        }
+      end
+    end
+  end
+end

lib/onebox/engine/typeform_onebox.rb

@@ -0,0 +1,48 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class TypeformOnebox
+      include Engine
+
+      matches_regexp(/^https?:\/\/[a-z0-9\-_]+\.typeform\.com\/to\/[a-zA-Z0-9]+/)
+      requires_iframe_origins "https://*.typeform.com"
+      always_https
+
+      def to_html
+        typeform_src = build_typeform_src
+
+        <<~HTML
+          <iframe
+            src="#{typeform_src}"
+            width="100%"
+            height="600px"
+            scrolling="no"
+            frameborder="0"
+          ></iframe>
+        HTML
+      end
+
+      def placeholder_html
+        ::Onebox::Helpers.generic_placeholder_html
+      end
+
+      private
+
+      def build_typeform_src
+        escaped_src = ::Onebox::Helpers.normalize_url_for_output(@url)
+        query_params = CGI::parse(URI::parse(escaped_src).query || '')
+
+        return escaped_src if query_params.has_key?('typeform-embed')
+
+        if query_params.empty?
+          escaped_src += '?' unless escaped_src.end_with?('?')
+        else
+          escaped_src += '&'
+        end
+
+        escaped_src += 'typeform-embed=embed-widget'
+      end
+    end
+  end
+end

lib/onebox/engine/video_onebox.rb

@@ -0,0 +1,36 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class VideoOnebox
+      include Engine
+
+      matches_regexp(/^(https?:)?\/\/.*\.(mov|mp4|webm|ogv)(\?.*)?$/i)
+
+      def always_https?
+        AllowlistedGenericOnebox.host_matches(uri, AllowlistedGenericOnebox.https_hosts)
+      end
+
+      def to_html
+        # Fix Dropbox image links
+        if @url[/^https:\/\/www.dropbox.com\/s\//]
+          @url.sub!("https://www.dropbox.com", "https://dl.dropboxusercontent.com")
+        end
+
+        escaped_url = ::Onebox::Helpers.normalize_url_for_output(@url)
+        <<-HTML
+          <div class="onebox video-onebox">
+            <video width='100%' height='100%' controls #{@options[:disable_media_download_controls] ? 'controlslist="nodownload"' : ""}>
+              <source src='#{escaped_url}'>
+              <a href='#{escaped_url}'>#{@url}</a>
+            </video>
+          </div>
+        HTML
+      end
+
+      def placeholder_html
+        ::Onebox::Helpers.video_placeholder_html
+      end
+    end
+  end
+end

lib/onebox/engine/vimeo_onebox.rb

@@ -0,0 +1,55 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class VimeoOnebox
+      include Engine
+      include StandardEmbed
+
+      matches_regexp(/^https?:\/\/(www\.)?vimeo\.com\/\d+/)
+      requires_iframe_origins "https://player.vimeo.com"
+      always_https
+
+      WIDTH  ||= 640
+      HEIGHT ||= 360
+
+      def placeholder_html
+        ::Onebox::Helpers.video_placeholder_html
+      end
+
+      def to_html
+        video_id = oembed_data[:video_id]
+        if video_id.nil?
+          # for private videos
+          video_id = uri.path[/\/(\d+)/, 1]
+        end
+        video_src = "https://player.vimeo.com/video/#{video_id}"
+        video_src = video_src.gsub('autoplay=1', '').chomp("?")
+
+        <<-HTML
+          <iframe
+            width="#{WIDTH}"
+            height="#{HEIGHT}"
+            src="#{video_src}"
+            data-original-href="#{link}"
+            frameborder="0"
+            allowfullscreen
+          ></iframe>
+        HTML
+      end
+
+      private
+
+      def oembed_data
+        response = Onebox::Helpers.fetch_response("https://vimeo.com/api/oembed.json?url=#{url}")
+        @oembed_data = Onebox::Helpers.symbolize_keys(::MultiJson.load(response))
+      rescue
+        "{}"
+      end
+
+      def og_data
+        @og_data = get_opengraph
+      end
+    end
+  end
+end

lib/onebox/engine/wikimedia_onebox.rb

@@ -0,0 +1,43 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class WikimediaOnebox
+      include Engine
+      include LayoutSupport
+      include JSON
+
+      matches_regexp(/^https?:\/\/commons\.wikimedia\.org\/wiki\/(File:.+)/)
+      always_https
+
+      def self.priority
+        # Wikimedia links end in an image extension.
+        # E.g. https://commons.wikimedia.org/wiki/File:Stones_members_montage2.jpg
+        # This engine should have priority over the generic ImageOnebox.
+
+        1
+      end
+
+      def url
+        "https://en.wikipedia.org/w/api.php?action=query&titles=#{match[:name]}&prop=imageinfo&iilimit=50&iiprop=timestamp|user|url&iiurlwidth=500&format=json"
+      end
+
+      private
+
+      def match
+        @match ||= @url.match(/^https?:\/\/commons\.wikimedia\.org\/wiki\/(?<name>File:.+)/)
+      end
+
+      def data
+        first_page = raw['query']['pages'].first[1]
+
+        {
+          link: first_page['imageinfo'].first['descriptionurl'],
+          title: first_page['title'],
+          image: first_page['imageinfo'].first['url'],
+          thumbnail: first_page['imageinfo'].first['thumburl']
+        }
+      end
+    end
+  end
+end

lib/onebox/engine/wikipedia_onebox.rb

@@ -0,0 +1,97 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class WikipediaOnebox
+      include Engine
+      include LayoutSupport
+      include HTML
+
+      matches_regexp(/^https?:\/\/.*\.wikipedia\.(com|org)/)
+      always_https
+
+      private
+
+      def data
+        paras = []
+        text = ""
+
+        # Detect section Hash in the url and retrive the related paragraphs. if no hash provided the first few paragraphs will be used
+        # Author Lidlanca
+        # Date 9/8/2014
+        if (m_url_hash = @url.match(/#([^\/?]+)/)) # extract url hash
+          m_url_hash_name = m_url_hash[1]
+        end
+
+        unless m_url_hash.nil?
+          section_header_title = raw.xpath("//span[@id='#{m_url_hash_name}']")
+
+          if section_header_title.empty?
+            paras = raw.search("p") # default get all the paras
+          else
+            section_title_text = section_header_title.inner_text
+            section_header = section_header_title[0].parent # parent element of the section span element should be an <h3> node
+            cur_element = section_header
+
+            # p|text|div covers the general case. We assume presence of at least 1 P node. if section has no P node we may end up with a P node from the next section.
+            # div tag is commonly used as an assets wraper in an article section. often as the first element holding an image.
+            # ul support will imporve the output generated for a section with a list as the main content (for example: an Author Bibliography, A musician Discography, etc)
+            first_p_found = nil
+            while (((next_sibling = cur_element.next_sibling).name =~ /p|text|div|ul/) || first_p_found.nil?) do  # from section header get the next sibling until it is a breaker tag
+              cur_element = next_sibling
+              if (cur_element.name == "p" || cur_element.name == "ul") #we treat a list as we detect a p to avoid showing
+                first_p_found = true
+                paras.push(cur_element)
+              end
+            end
+          end
+        else # no hash found in url
+          paras = raw.search("p") # default get all the paras
+        end
+
+        unless paras.empty?
+          cnt = 0
+          while text.length < Onebox::LayoutSupport.max_text && cnt <= 3
+            break if cnt >= paras.size
+            text += " " unless cnt == 0
+
+            if paras[cnt].name == "ul" # Handle UL tag. Generate a textual ordered list (1.item | 2.item | 3.item). Unfortunately no newline allowed in output
+              li_index = 1
+              list_items = []
+              paras[cnt].children.css("li").each { |li| list_items.push "#{li_index}." + li.inner_text ; li_index += 1 }
+              paragraph = (list_items.join " |\n ")[0..Onebox::LayoutSupport.max_text]
+            else
+              paragraph = paras[cnt].inner_text[0..Onebox::LayoutSupport.max_text]
+            end
+
+            paragraph.gsub!(/\[\d+\]/mi, "")
+            text += paragraph
+            cnt += 1
+          end
+        end
+
+        text = "#{text[0..Onebox::LayoutSupport.max_text]}..." if text.length > Onebox::LayoutSupport.max_text
+
+        result = {
+          link: link,
+          title: raw.css("html body h1").inner_text + (section_title_text ? " | " + section_title_text : ""),  #if a section sub title exists add it to the main article title
+          description: text
+        }
+
+        img = raw.css(".image img")
+
+        if img && img.size > 0
+          img.each do |i|
+            src = i["src"]
+            if src !~ /Question_book/
+              result[:image] = src
+              break
+            end
+          end
+        end
+
+        result
+      end
+    end
+  end
+end

lib/onebox/engine/wistia_onebox.rb

@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class WistiaOnebox
+      include Engine
+      include StandardEmbed
+
+      matches_regexp(/https?:\/\/(.+)?(wistia.com|wi.st)\/(medias|embed)\/.*/)
+      requires_iframe_origins("https://fast.wistia.com", "https://fast.wistia.net")
+      always_https
+
+      def to_html
+        get_oembed.html
+      end
+
+      def placeholder_html
+        oembed = get_oembed
+        return if Onebox::Helpers.blank?(oembed.thumbnail_url)
+        "<img src='#{oembed.thumbnail_url}' #{oembed.title_attr}>"
+      end
+
+      private
+
+      def get_oembed_url
+        "https://fast.wistia.com/oembed?embedType=iframe&url=#{url}"
+      end
+    end
+  end
+end

lib/onebox/engine/xkcd_onebox.rb

@@ -0,0 +1,32 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class XkcdOnebox
+      include Engine
+      include LayoutSupport
+      include JSON
+
+      matches_regexp(/^https?:\/\/(www\.)?(m\.)?xkcd\.com\/\d+/)
+
+      def url
+        "https://xkcd.com/#{match[:comic_id]}/info.0.json"
+      end
+
+      private
+
+      def match
+        @match ||= @url.match(%{xkcd\.com/(?<comic_id>\\d+)})
+      end
+
+      def data
+        {
+          link: @url,
+          title: raw['safe_title'],
+          image: raw['img'],
+          description: raw['alt']
+        }
+      end
+    end
+  end
+end

lib/onebox/engine/youku_onebox.rb

@@ -0,0 +1,35 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class YoukuOnebox
+      include Engine
+      include HTML
+
+      matches_regexp(/^(https?:\/\/)?([\da-z\.-]+)(youku.com\/)(.)+\/?$/)
+      requires_iframe_origins "https://player.youku.com"
+
+      # Try to get the video ID. Works for URLs of the form:
+      # * http://v.youku.com/v_show/id_XNjM3MzAxNzc2.html
+      # * http://v.youku.com/v_show/id_XMTQ5MjgyMjMyOA==.html?from=y1.3-tech-index3-232-10183.89969-89963.3-1
+      def video_id
+        match = uri.path.match(/\/v_show\/id_([a-zA-Z0-9_=\-]+)(\.html)?.*/)
+        match && match[1]
+      rescue
+        nil
+      end
+
+      def to_html
+        <<~HTML
+          <iframe
+            src="https://player.youku.com/embed/#{video_id}"
+            width="640"
+            height="430"
+            frameborder='0'
+            allowfullscreen
+          ></iframe>
+        HTML
+      end
+    end
+  end
+end

lib/onebox/engine/youtube_onebox.rb

@@ -0,0 +1,173 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Engine
+    class YoutubeOnebox
+      include Engine
+      include StandardEmbed
+
+      matches_regexp(/^https?:\/\/(?:www\.)?(?:m\.)?(?:youtube\.com|youtu\.be)\/.+$/)
+      requires_iframe_origins "https://www.youtube.com"
+      always_https
+
+      WIDTH  ||= 480
+      HEIGHT ||= 360
+
+      def parse_embed_response
+        return unless video_id
+        return @parse_embed_response if defined?(@parse_embed_response)
+
+        embed_url = "https://www.youtube.com/embed/#{video_id}"
+        @embed_doc ||= Onebox::Helpers.fetch_html_doc(embed_url)
+
+        begin
+          script_tag = @embed_doc.xpath('//script').find { |tag| tag.to_s.include?('ytcfg.set') }.to_s
+          match = script_tag.to_s.match(/ytcfg\.set\((?<json>.*)\)/)
+
+          yt_json = ::JSON.parse(match[:json])
+          renderer = ::JSON.parse(yt_json['PLAYER_VARS']['embedded_player_response'])['embedPreview']['thumbnailPreviewRenderer']
+
+          title = renderer['title']['runs'].first['text']
+
+          image = "https://img.youtube.com/vi/#{video_id}/hqdefault.jpg"
+        rescue
+          return
+        end
+
+        @parse_embed_response = { image: image, title: title }
+      end
+
+      def placeholder_html
+        if video_id || list_id
+          result = parse_embed_response
+          result ||= get_opengraph.data
+
+          "<img src='#{result[:image]}' width='#{WIDTH}' height='#{HEIGHT}' title='#{result[:title]}'>"
+        else
+          to_html
+        end
+      end
+
+      def to_html
+        if video_id
+          <<-HTML
+            <iframe
+              src="https://www.youtube.com/embed/#{video_id}?#{embed_params}"
+              width="#{WIDTH}"
+              height="#{HEIGHT}"
+              frameborder="0"
+              allowfullscreen
+            ></iframe>
+          HTML
+        elsif list_id
+          <<-HTML
+            <iframe
+              src="https://www.youtube.com/embed/videoseries?list=#{list_id}&wmode=transparent&rel=0&autohide=1&showinfo=1&enablejsapi=1"
+              width="#{WIDTH}"
+              height="#{HEIGHT}"
+              frameborder="0"
+              allowfullscreen
+            ></iframe>
+          HTML
+        else
+          # for channel pages
+          html = Onebox::Engine::AllowlistedGenericOnebox.new(@url, @timeout).to_html
+          return if Onebox::Helpers.blank?(html)
+          html.gsub!(/['"]\/\//, "https://")
+          html
+        end
+      end
+
+      def video_title
+        @video_title ||= begin
+          result = parse_embed_response || get_opengraph.data
+          result[:title]
+        end
+      end
+
+      private
+
+      def video_id
+        @video_id ||= begin
+          # http://youtu.be/afyK1HSFfgw
+          if uri.host["youtu.be"]
+            id = uri.path[/\/([\w\-]+)/, 1]
+            return id if id
+          end
+
+          # https://www.youtube.com/embed/vsF0K3Ou1v0
+          if uri.path["/embed/"]
+            id = uri.path[/\/embed\/([\w\-]+)/, 1]
+            return id if id
+          end
+
+          # https://www.youtube.com/watch?v=Z0UISCEe52Y
+          params['v']
+        end
+      end
+
+      def list_id
+        @list_id ||= params['list']
+      end
+
+      def embed_params
+        p = { 'feature' => 'oembed', 'wmode' => 'opaque' }
+
+        p['list'] = list_id if list_id
+
+        # Parse timestrings, and assign the result as a start= parameter
+        start = if params['start']
+          params['start']
+        elsif params['t']
+          params['t']
+        elsif uri.fragment && uri.fragment.start_with?('t=')
+          # referencing uri is safe here because any throws were already caught by video_id returning nil
+          # remove the t= from the start
+          uri.fragment[2..-1]
+        end
+
+        p['start'] = parse_timestring(start)        if start
+        p['end']   = parse_timestring params['end'] if params['end']
+
+        # Official workaround for looping videos
+        # https://developers.google.com/youtube/player_parameters#loop
+        # use params.include? so that you can just add "&loop"
+        if params.include?('loop')
+          p['loop'] = 1
+          p['playlist'] = video_id
+        end
+
+        # https://developers.google.com/youtube/player_parameters#rel
+        p['rel'] = 0 if params.include?('rel')
+
+        # https://developers.google.com/youtube/player_parameters#enablejsapi
+        p['enablejsapi'] = params['enablejsapi'] if params.include?('enablejsapi')
+
+        URI.encode_www_form(p)
+      end
+
+      def parse_timestring(string)
+        if string =~ /(\d+h)?(\d+m)?(\d+s?)?/
+          ($1.to_i * 3600) + ($2.to_i * 60) + $3.to_i
+        end
+      end
+
+      def params
+        return {} unless uri.query
+        # This mapping is necessary because CGI.parse returns a hash of keys to arrays.
+        # And *that* is necessary because querystrings support arrays, so they
+        # force you to deal with it to avoid security issues that would pop up
+        # if one day it suddenly gave you an array.
+        #
+        # However, we aren't interested. Just take the first one.
+        @params ||= begin
+          p = {}
+          CGI.parse(uri.query).each { |k, v| p[k] = v.first }
+          p
+        end
+      rescue
+        {}
+      end
+    end
+  end
+end

lib/onebox/file_type_finder.rb

@@ -0,0 +1,70 @@
+# frozen_string_literal: true
+
+module Onebox
+  module FileTypeFinder
+    # In general, most of file extension names would be recognized
+    # by Highlights.js.  However, some need to be checked in other
+    # ways, either because they just aren't included, because they
+    # are extensionless, or because they contain dots (they are
+    # multi-part).
+    # IMPORTANT: to prevent false positive matching, start all
+    # entries on this list with a "."
+    #
+    # For easy reference, keep these sorted in alphabetical order.
+    @long_file_types = {
+      ".bib" => "tex",
+      ".html.hbs" => "handlebars",
+      ".html.handlebars" => "handlebars",
+      ".latex" => "tex",
+      ".ru" => "rb",
+      ".simplecov" => "rb", # Not official, but seems commonly found
+      ".sty" => "tex"
+    }
+
+    # Some extensionless files for which we know the type
+    # These should all be stored LOWERCASE, just for consistency.
+    # The ones that I know of also include the ".lock" fake extension.
+    #
+    # For easy reference, keep these sorted in alphabetical order,
+    # FIRST by their types and THEN by their names.
+    @extensionless_files = {
+      "cmake.in" => "cmake",
+
+      "gruntfile" => "js",
+      "gulpfile" => "js",
+
+      "artisan" => "php",
+
+      "berksfile" => "rb",
+      "capfile" => "rb",
+      "cheffile" => "rb",
+      "cheffile.lock" => "rb",
+      "gemfile" => "rb",
+      "guardfile" => "rb",
+      "rakefile" => "rb",
+      "thorfile" => "rb",
+      "vagrantfile" => "rb",
+
+      "boxfile" => "yaml" # Not currently (2014-11) in Highlight.js
+    }
+
+    def self.from_file_name(file_name)
+      lower_name = file_name.downcase
+      # First check against the known lists of "special" files and extensions.
+      return @extensionless_files[lower_name] if @extensionless_files.has_key?(lower_name)
+
+      @long_file_types.each { |extension, type|
+        return type if lower_name.end_with?(extension)
+      }
+
+      # Otherwise, just split on the last ".",
+      # but add one so we don't return the "." itself.
+      dot_spot = lower_name.rindex(".")
+      return lower_name[(dot_spot + 1)..-1] if dot_spot
+
+      # If we couldn't figure it out from the name,
+      # let the highlighter figure it out from the content.
+      ""
+    end
+  end
+end

lib/onebox/helpers.rb

@@ -0,0 +1,252 @@
+# frozen_string_literal: true
+
+require "addressable"
+
+module Onebox
+  module Helpers
+
+    class DownloadTooLarge < StandardError; end
+
+    IGNORE_CANONICAL_DOMAINS ||= ['www.instagram.com', 'youtube.com']
+
+    def self.symbolize_keys(hash)
+      return {} if hash.nil?
+
+      hash.inject({}) do |result, (key, value)|
+        new_key = key.is_a?(String) ? key.to_sym : key
+        new_value = value.is_a?(Hash) ? symbolize_keys(value) : value
+        result[new_key] = new_value
+        result
+      end
+    end
+
+    def self.clean(html)
+      html.gsub(/<[^>]+>/, ' ').gsub(/\n/, '')
+    end
+
+    def self.fetch_html_doc(url, headers = nil, body_cacher = nil)
+      response = (fetch_response(url, headers: headers, body_cacher: body_cacher) rescue nil)
+      doc = Nokogiri::HTML(response)
+      uri = Addressable::URI.parse(url)
+
+      ignore_canonical_tag = doc.at('meta[property="og:ignore_canonical"]')
+      should_ignore_canonical = IGNORE_CANONICAL_DOMAINS.map { |hostname| uri.hostname.match?(hostname) }.any?
+
+      unless (ignore_canonical_tag && ignore_canonical_tag['content'].to_s == 'true') || should_ignore_canonical
+        # prefer canonical link
+        canonical_link = doc.at('//link[@rel="canonical"]/@href')
+        canonical_uri = Addressable::URI.parse(canonical_link)
+        if canonical_link && "#{canonical_uri.host}#{canonical_uri.path}" != "#{uri.host}#{uri.path}"
+          response = (fetch_response(canonical_uri.to_s, headers: headers, body_cacher: body_cacher) rescue nil)
+          doc = Nokogiri::HTML(response) if response
+        end
+      end
+
+      doc
+    end
+
+    def self.fetch_response(location, redirect_limit: 5, domain: nil, headers: nil, body_cacher: nil)
+      redirect_limit = Onebox.options.redirect_limit if redirect_limit > Onebox.options.redirect_limit
+
+      raise Net::HTTPError.new('HTTP redirect too deep', location) if redirect_limit == 0
+
+      uri = Addressable::URI.parse(location)
+      uri = Addressable::URI.join(domain, uri) if !uri.host
+
+      use_body_cacher = body_cacher && body_cacher.respond_to?('fetch_cached_response_body')
+      if use_body_cacher
+        response_body = body_cacher.fetch_cached_response_body(uri.to_s)
+
+        if response_body.present?
+          return response_body
+        end
+      end
+
+      result = StringIO.new
+      Net::HTTP.start(uri.host, uri.port, use_ssl: uri.normalized_scheme == 'https') do |http|
+        http.open_timeout = Onebox.options.connect_timeout
+        http.read_timeout = Onebox.options.timeout
+        http.verify_mode = OpenSSL::SSL::VERIFY_NONE  # Work around path building bugs
+
+        headers ||= {}
+
+        if Onebox.options.user_agent && !headers['User-Agent']
+          headers['User-Agent'] = Onebox.options.user_agent
+        end
+
+        request = Net::HTTP::Get.new(uri.request_uri, headers)
+        start_time = Time.now
+
+        size_bytes = Onebox.options.max_download_kb * 1024
+        http.request(request) do |response|
+
+          if cookie = response.get_fields('set-cookie')
+            # HACK: If this breaks again in the future, use HTTP::CookieJar from gem 'http-cookie'
+            # See test: it "does not send cookies to the wrong domain"
+            redir_header = { 'Cookie' => cookie.join('; ') }
+          end
+
+          redir_header = nil unless redir_header.is_a? Hash
+
+          code = response.code.to_i
+          unless code === 200
+            response.error! unless [301, 302, 303, 307, 308].include?(code)
+
+            return fetch_response(
+              response['location'],
+              redirect_limit: redirect_limit - 1,
+              domain: "#{uri.scheme}://#{uri.host}",
+              headers: redir_header
+            )
+          end
+
+          response.read_body do |chunk|
+            result.write(chunk)
+            raise DownloadTooLarge.new if result.size > size_bytes
+            raise Timeout::Error.new if (Time.now - start_time) > Onebox.options.timeout
+          end
+
+          if use_body_cacher && body_cacher.cache_response_body?(uri)
+            body_cacher.cache_response_body(uri.to_s, result.string)
+          end
+
+          return result.string
+        end
+      end
+    end
+
+    def self.fetch_content_length(location)
+      uri = URI(location)
+
+      Net::HTTP.start(uri.host, uri.port, use_ssl: uri.is_a?(URI::HTTPS)) do |http|
+        http.open_timeout = Onebox.options.connect_timeout
+        http.read_timeout = Onebox.options.timeout
+        if uri.is_a?(URI::HTTPS)
+          http.use_ssl = true
+          http.verify_mode = OpenSSL::SSL::VERIFY_NONE
+        end
+
+        http.request_head([uri.path, uri.query].join("?")) do |response|
+          code = response.code.to_i
+          unless code === 200 || Onebox::Helpers.blank?(response.content_length)
+            return nil
+          end
+          return response.content_length
+        end
+      end
+    end
+
+    def self.pretty_filesize(size)
+      conv = [ 'B', 'KB', 'MB', 'GB', 'TB', 'PB', 'EB' ]
+      scale = 1024
+
+      ndx = 1
+      if (size < 2 * (scale**ndx)) then
+        return "#{(size)} #{conv[ndx - 1]}"
+      end
+      size = size.to_f
+      [2, 3, 4, 5, 6, 7].each do |i|
+        if (size < 2 * (scale**i)) then
+          return "#{'%.2f' % (size / (scale**(i - 1)))} #{conv[i - 1]}"
+        end
+      end
+      ndx = 7
+      "#{'%.2f' % (size / (scale**(ndx - 1)))} #{conv[ndx - 1]}"
+    end
+
+    def self.click_to_scroll_div(width = 690, height = 400)
+      "<div style=\"background:transparent;position:relative;width:#{width}px;height:#{height}px;top:#{height}px;margin-top:-#{height}px;\" onClick=\"style.pointerEvents='none'\"></div>"
+    end
+
+    def self.blank?(value)
+      if value.nil?
+        true
+      elsif String === value
+        value.empty? || !(/[[:^space:]]/ === value)
+      else
+        value.respond_to?(:empty?) ? !!value.empty? : !value
+      end
+    end
+
+    def self.truncate(string, length = 50)
+      return string if string.nil?
+      string.size > length ? string[0...(string.rindex(" ", length) || length)] + "..." : string
+    end
+
+    def self.get(meta, attr)
+      (meta && !blank?(meta[attr])) ? sanitize(meta[attr]) : nil
+    end
+
+    def self.sanitize(value, length = 50)
+      return nil if blank?(value)
+      Sanitize.fragment(value).strip
+    end
+
+    def self.normalize_url_for_output(url)
+      return "" unless url
+      url = url.dup
+      # expect properly encoded url, remove any unsafe chars
+      url.gsub!(' ', '%20')
+      url.gsub!("'", "&apos;")
+      url.gsub!('"', "&quot;")
+      url.gsub!(/[^\w\-`.~:\/?#\[\]@!$&'\(\)*+,;=%\p{M}’]/, "")
+
+      parsed = Addressable::URI.parse(url)
+      return "" unless parsed.host
+
+      url
+    end
+
+    def self.get_absolute_image_url(src, url)
+      if src && !!(src =~ /^\/\//)
+        uri = URI(url)
+        src = "#{uri.scheme}:#{src}"
+      elsif src && src.match(/^https?:\/\//i).nil?
+        uri = URI(url)
+        src = if !src.start_with?("/") && uri.path.present?
+          "#{uri.scheme}://#{uri.host.sub(/\/$/, '')}#{uri.path.sub(/\/$/, '')}/#{src.sub(/^\//, '')}"
+        else
+          "#{uri.scheme}://#{uri.host.sub(/\/$/, '')}/#{src.sub(/^\//, '')}"
+        end
+      end
+      src
+    end
+
+    # Percent-encodes a URI string per RFC3986 - https://tools.ietf.org/html/rfc3986
+    def self.uri_encode(url)
+      return "" unless url
+
+      uri = Addressable::URI.parse(url)
+
+      encoded_uri = Addressable::URI.new(
+        scheme: Addressable::URI.encode_component(uri.scheme, Addressable::URI::CharacterClasses::SCHEME),
+        authority: Addressable::URI.encode_component(uri.authority, Addressable::URI::CharacterClasses::AUTHORITY),
+        path: Addressable::URI.encode_component(uri.path, Addressable::URI::CharacterClasses::PATH + "\\%"),
+        query: Addressable::URI.encode_component(uri.query, "a-zA-Z0-9\\-\\.\\_\\~\\$\\&\\*\\,\\=\\:\\@\\?\\%"),
+        fragment: Addressable::URI.encode_component(uri.fragment, "a-zA-Z0-9\\-\\.\\_\\~\\!\\$\\&\\'\\(\\)\\*\\+\\,\\;\\=\\:\\/\\?\\%")
+      )
+
+      encoded_uri.to_s
+    end
+
+    def self.uri_unencode(url)
+      Addressable::URI.unencode(url)
+    end
+
+    def self.video_placeholder_html
+      "<div class='onebox-placeholder-container'><span class='placeholder-icon video'></span></div>"
+    end
+
+    def self.audio_placeholder_html
+      "<div class='onebox-placeholder-container'><span class='placeholder-icon audio'></span></div>"
+    end
+
+    def self.map_placeholder_html
+      "<div class='onebox-placeholder-container'><span class='placeholder-icon map'></span></div>"
+    end
+
+    def self.generic_placeholder_html
+      "<div class='onebox-placeholder-container'><span class='placeholder-icon generic'></span></div>"
+    end
+  end
+end

lib/onebox/layout.rb

@@ -0,0 +1,55 @@
+# frozen_string_literal: true
+
+require_relative "template_support"
+
+module Onebox
+  class Layout < Mustache
+    include TemplateSupport
+
+    VERSION = "1.0.0"
+
+    attr_reader :record
+    attr_reader :view
+
+    def initialize(name, record)
+      @record = Onebox::Helpers.symbolize_keys(record)
+
+      # Fix any relative paths
+      if @record[:image] && @record[:image] =~ /^\/[^\/]/
+        @record[:image] = "#{uri.scheme}://#{uri.host}/#{@record[:image]}"
+      end
+
+      @md5 = Digest::MD5.new
+      @view = View.new(name, @record)
+      @template_name = "_layout"
+      @template_path = load_paths.last
+    end
+
+    def to_html
+      render(details)
+    end
+
+    private
+
+    def uri
+      @uri ||= URI(::Onebox::Helpers.normalize_url_for_output(record[:link]))
+    end
+
+    def details
+      {
+        link: record[:link],
+        title: record[:title],
+        favicon: record[:favicon],
+        domain: record[:domain] || uri.host.to_s.sub(/^www\./, ''),
+        article_published_time: record[:article_published_time],
+        article_published_time_title: record[:article_published_time_title],
+        metadata_1_label: record[:metadata_1_label],
+        metadata_1_value: record[:metadata_1_value],
+        metadata_2_label: record[:metadata_2_label],
+        metadata_2_value: record[:metadata_2_value],
+        subname: view.template_name,
+        view: view.to_html
+      }
+    end
+  end
+end

lib/onebox/layout_support.rb

@@ -0,0 +1,18 @@
+# frozen_string_literal: true
+
+module Onebox
+  module LayoutSupport
+
+    def self.max_text
+      500
+    end
+
+    def layout
+      @layout ||= Layout.new(self.class.onebox_name, data)
+    end
+
+    def to_html
+      layout.to_html
+    end
+  end
+end

lib/onebox/matcher.rb

@@ -0,0 +1,32 @@
+# frozen_string_literal: true
+
+module Onebox
+  class Matcher
+    def initialize(url, options = {})
+      begin
+        @uri = URI(url)
+      rescue URI::InvalidURIError
+      end
+
+      @options = options
+    end
+
+    def ordered_engines
+      @ordered_engines ||= Engine.engines.sort_by do |e|
+        e.respond_to?(:priority) ? e.priority : 100
+      end
+    end
+
+    def oneboxed
+      return if @uri.nil?
+      return if @uri.port && !Onebox.options.allowed_ports.include?(@uri.port)
+      return if @uri.scheme && !Onebox.options.allowed_schemes.include?(@uri.scheme)
+      ordered_engines.find { |engine| engine === @uri && has_allowed_iframe_origins?(engine) }
+    end
+
+    def has_allowed_iframe_origins?(engine)
+      allowed_regexes = @options[:allowed_iframe_regexes] || []
+      engine.iframe_origins.all? { |o| allowed_regexes.any? { |r| o =~ r } }
+    end
+  end
+end

lib/onebox/mixins/git_blob_onebox.rb

@@ -0,0 +1,228 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Mixins
+    module GitBlobOnebox
+      def self.included(klass)
+        klass.include(Onebox::Engine)
+        klass.include(Onebox::LayoutSupport)
+        klass.matches_regexp(klass.git_regexp)
+        klass.always_https
+        klass.include(InstanceMethods)
+      end
+
+      EXPAND_AFTER = 0b001
+      EXPAND_BEFORE = 0b010
+      EXPAND_NONE = 0b0
+
+      DEFAULTS = {
+        EXPAND_ONE_LINER: EXPAND_AFTER | EXPAND_BEFORE, #set how to expand a one liner. user EXPAND_NONE to disable expand
+        LINES_BEFORE: 10,
+        LINES_AFTER: 10,
+        SHOW_LINE_NUMBER: true,
+        MAX_LINES: 20,
+        MAX_CHARS: 5000
+      }
+
+      module InstanceMethods
+        def initialize(url, timeout = nil)
+          super url, timeout
+          # merge engine options from global Onebox.options interface
+          # self.options = Onebox.options["GithubBlobOnebox"] #  self.class.name.split("::").last.to_s
+          # self.options = Onebox.options[self.class.name.split("::").last.to_s] #We can use this a more generic approach. extract the engine class name automatically
+
+          self.options = DEFAULTS
+
+          @selected_lines_array = nil
+          @selected_one_liner = 0
+          @model_file = nil
+
+          # Define constant after merging options set in Onebox.options
+          # We can define constant automatically.
+          options.each_pair do |constant_name, value|
+            constant_name_u = constant_name.to_s.upcase
+            if constant_name_u == constant_name.to_s
+              #define a constant if not already defined
+              unless self.class.const_defined? constant_name_u.to_sym
+                Onebox::Mixins::GitBlobOnebox.const_set constant_name_u.to_sym , options[constant_name_u.to_sym]
+              end
+            end
+          end
+        end
+
+        private
+
+        def calc_range(m, contents_lines_size)
+          truncated = false
+          from = /\d+/.match(m[:from])             #get numeric should only match a positive interger
+          to   = /\d+/.match(m[:to])               #get numeric should only match a positive interger
+          range_provided = !(from.nil? && to.nil?) #true if "from" or "to" provided in URL
+          from = from.nil? ?  1 : from[0].to_i     #if from not provided default to 1st line
+          to   = to.nil?   ? -1 : to[0].to_i       #if to not provided default to undefiend to be handled later in the logic
+
+          if to === -1 && range_provided   #case "from" exists but no valid "to". aka ONE_LINER
+            one_liner = true
+            to = from
+          else
+            one_liner = false
+          end
+
+          unless range_provided  #case no range provided default to 1..MAX_LINES
+            from = 1
+            to   = MAX_LINES
+            truncated = true if contents_lines_size > MAX_LINES
+            #we can technically return here
+          end
+
+          from, to = [from, to].sort                                #enforce valid range.  [from < to]
+          from = 1 if from > contents_lines_size                   #if "from" out of TOP bound set to 1st line
+          to   = contents_lines_size if to > contents_lines_size   #if "to" is out of TOP bound set to last line.
+
+          if one_liner
+            @selected_one_liner = from
+            if EXPAND_ONE_LINER != EXPAND_NONE
+              if (EXPAND_ONE_LINER & EXPAND_BEFORE != 0) # check if EXPAND_BEFORE flag is on
+                from = [1, from - LINES_BEFORE].max      # make sure expand before does not go out of bound
+              end
+
+              if (EXPAND_ONE_LINER & EXPAND_AFTER != 0)          # check if EXPAND_FLAG flag is on
+                to = [to + LINES_AFTER, contents_lines_size].min # make sure expand after does not go out of bound
+              end
+
+              from = contents_lines_size if from > contents_lines_size   #if "from" is out of the content top bound
+              # to   = contents_lines_size if to > contents_lines_size   #if "to" is out of  the content top bound
+            else
+              #no expand show the one liner solely
+            end
+          end
+
+          if to - from > MAX_LINES && !one_liner  #if exceed the MAX_LINES limit correct unless range was produced by one_liner which it expand setting will allow exceeding the line limit
+            truncated = true
+           to = from + MAX_LINES - 1
+          end
+
+          {
+            from: from,                               #calculated from
+            from_minus_one: from - 1,                   #used for getting currect ol>li numbering with css used in template
+            to: to,                                   #calculated to
+            one_liner: one_liner,                     #boolean if a one-liner
+            selected_one_liner: @selected_one_liner,  #if a one liner is provided we create a reference for it.
+            range_provided: range_provided,           #boolean if range provided
+            truncated: truncated
+          }
+        end
+
+        #minimize/compact leading indentation while preserving overall indentation
+        def removeLeadingIndentation(str)
+          min_space = 100
+          a_lines = str.lines
+          a_lines.each do |l|
+            l = l.chomp("\n")  # remove new line
+            m = l.match(/^[ ]*/) # find leading spaces 0 or more
+            unless m.nil? || l.size == m[0].size || l.size == 0 # no match | only spaces in line | empty line
+              m_str_length = m[0].size
+              if m_str_length <= 1  # minimum space is 1 or nothing we can break we found our minimum
+                min_space = m_str_length
+                break #stop iteration
+              end
+              if m_str_length < min_space
+                min_space = m_str_length
+              end
+            else
+              next # SKIP no match or line is only spaces
+            end
+          end
+          a_lines.each do |l|
+            re = Regexp.new "^[ ]{#{min_space}}"  #match the minimum spaces of the line
+            l.gsub!(re, "")
+          end
+          a_lines.join
+        end
+
+        def line_number_helper(lines, start, selected)
+          lines = removeLeadingIndentation(lines.join).lines # A little ineffeicent we could modify  removeLeadingIndentation to accept array and return array, but for now it is only working with a string
+          hash_builder = []
+          output_builder = []
+          lines.map.with_index { |line, i|
+            lnum = (i.to_i + start)
+            hash_builder.push(line_number: lnum, data: line.gsub("\n", ""), selected: (selected == lnum) ? true : false)
+            output_builder.push "#{lnum}: #{line}"
+          }
+          { output: output_builder.join(), array: hash_builder }
+        end
+
+        def raw
+          return @raw if defined?(@raw)
+
+          m = @url.match(self.raw_regexp)
+
+          if m
+            from = /\d+/.match(m[:from])   #get numeric should only match a positive interger
+            to   = /\d+/.match(m[:to])     #get numeric should only match a positive interger
+
+            @file = m[:file]
+            @lang = Onebox::FileTypeFinder.from_file_name(m[:file])
+
+            if @lang == "stl" && link.match?(/^https?:\/\/(www\.)?github\.com.*\/blob\//)
+              @model_file = @lang.dup
+              @raw = "https://render.githubusercontent.com/view/solid?url=" + self.raw_template(m)
+            else
+              contents = URI.open(self.raw_template(m), read_timeout: timeout).read
+
+              contents_lines = contents.lines           #get contents lines
+              contents_lines_size = contents_lines.size #get number of lines
+
+              cr = calc_range(m, contents_lines_size)    #calculate the range of lines for output
+              selected_one_liner = cr[:selected_one_liner] #if url is a one-liner calc_range will return it
+              from = cr[:from]
+              to = cr[:to]
+              @truncated = cr[:truncated]
+              range_provided = cr[:range_provided]
+              @cr_results = cr
+
+              if range_provided       #if a range provided (single line or more)
+                if SHOW_LINE_NUMBER
+                  lines_result = line_number_helper(contents_lines[(from - 1)..(to - 1)], from, selected_one_liner)  #print code with prefix line numbers in case range provided
+                  contents = lines_result[:output]
+                  @selected_lines_array = lines_result[:array]
+                else
+                  contents = contents_lines[(from - 1)..(to - 1)].join()
+                end
+
+              else
+                contents = contents_lines[(from - 1)..(to - 1)].join()
+              end
+
+              if contents.length > MAX_CHARS    #truncate content chars to limits
+                contents = contents[0..MAX_CHARS]
+                @truncated = true
+              end
+
+              @raw = contents
+            end
+          end
+        end
+
+        def data
+          @data ||= {
+            title: title,
+            link: link,
+            # IMPORTANT NOTE: All of the other class variables are populated
+            #     as *side effects* of the `raw` method! They must all appear
+            #     AFTER the call to `raw`! Don't get bitten by this like I did!
+            content: raw,
+            lang: "lang-#{@lang}",
+            lines: @selected_lines_array ,
+            has_lines: !@selected_lines_array.nil?,
+            selected_one_liner: @selected_one_liner,
+            cr_results: @cr_results,
+            truncated: @truncated,
+            model_file: @model_file,
+            width: 480,
+            height: 360
+          }
+        end
+      end
+    end
+  end
+end

lib/onebox/mixins/github_body.rb

@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Mixins
+    module GithubBody
+      def self.included(klass)
+        klass.include(Onebox::Engine)
+        klass.include(InstanceMethods)
+      end
+
+      module InstanceMethods
+        GITHUB_COMMENT_REGEX = /(<!--.*?-->\r\n)/
+        MAX_BODY_LENGTH = 80
+        def compute_body(body)
+          body = body.dup
+          excerpt = nil
+
+          body = (body || '').gsub(GITHUB_COMMENT_REGEX, '')
+          body = body.length > 0 ? body : nil
+          if body && body.length > MAX_BODY_LENGTH
+            excerpt = body[MAX_BODY_LENGTH..body.length].rstrip
+            body = body[0..MAX_BODY_LENGTH - 1]
+          end
+
+          [body, excerpt]
+        end
+      end
+    end
+  end
+end

lib/onebox/mixins/twitch_onebox.rb

@@ -0,0 +1,34 @@
+# frozen_string_literal: true
+
+module Onebox
+  module Mixins
+    module TwitchOnebox
+      def self.included(klass)
+        klass.include(Onebox::Engine)
+        klass.matches_regexp(klass.twitch_regexp)
+        klass.requires_iframe_origins "https://player.twitch.tv"
+        klass.include(InstanceMethods)
+      end
+
+      module InstanceMethods
+        def twitch_id
+          @url.match(self.class.twitch_regexp)[1]
+        end
+
+        def base_url
+          "player.twitch.tv/?"
+        end
+
+        def placeholder_html
+          ::Onebox::Helpers.video_placeholder_html
+        end
+
+        def to_html
+          <<~HTML
+          <iframe src="https://#{base_url}#{query_params}&parent=#{options[:hostname]}&autoplay=false" width="620" height="378" frameborder="0" style="overflow: hidden;" scrolling="no" allowfullscreen="allowfullscreen"></iframe>
+          HTML
+        end
+      end
+    end
+  end
+end

lib/onebox/oembed.rb

@@ -0,0 +1,17 @@
+# frozen_string_literal: true
+
+module Onebox
+  class Oembed < OpenGraph
+
+    def initialize(response)
+      @data = Onebox::Helpers.symbolize_keys(::MultiJson.load(response))
+
+      # never use oembed from WordPress 4.4 (it's broken)
+      data.delete(:html) if data[:html] && data[:html]["wp-embedded-content"]
+    end
+
+    def html
+      get(:html, nil, false)
+    end
+  end
+end

lib/onebox/open_graph.rb

@@ -0,0 +1,93 @@
+# frozen_string_literal: true
+
+module Onebox
+  class OpenGraph
+
+    attr_reader :data
+
+    def initialize(doc)
+      @data = extract(doc)
+    end
+
+    def title
+      get(:title, 80)
+    end
+
+    def title_attr
+      !title.nil? ? "title='#{title}'" : ""
+    end
+
+    def secure_image_url
+      secure_url = URI(get(:image))
+      secure_url.scheme = 'https'
+      secure_url.to_s
+    end
+
+    def method_missing(attr, *args, &block)
+      value = get(attr, *args)
+
+      return nil if Onebox::Helpers::blank?(value)
+
+      method_name = attr.to_s
+      if method_name.end_with?(*integer_suffixes)
+        value.to_i
+      elsif method_name.end_with?(*url_suffixes)
+        result = Onebox::Helpers.normalize_url_for_output(value)
+        result unless Onebox::Helpers::blank?(result)
+      else
+        value
+      end
+    end
+
+    def get(attr, length = nil, sanitize = true)
+      return nil if Onebox::Helpers::blank?(data)
+
+      value = data[attr]
+
+      return nil if Onebox::Helpers::blank?(value)
+
+      value = html_entities.decode(value)
+      value = Sanitize.fragment(value) if sanitize
+      value.strip!
+      value = Onebox::Helpers.truncate(value, length) unless length.nil?
+
+      value
+    end
+
+    private
+
+    def integer_suffixes
+      ['width', 'height']
+    end
+
+    def url_suffixes
+      ['url', 'image', 'video']
+    end
+
+    def html_entities
+      @html_entities ||= HTMLEntities.new
+    end
+
+    def extract(doc)
+      return {} if Onebox::Helpers::blank?(doc)
+
+      data = {}
+
+      doc.css('meta').each do |m|
+        if (m["property"] && m["property"][/^(?:og|article|product):(.+)$/i]) || (m["name"] && m["name"][/^(?:og|article|product):(.+)$/i])
+          value = (m["content"] || m["value"]).to_s
+          data[$1.tr('-:', '_').to_sym] ||= value unless Onebox::Helpers::blank?(value)
+        end
+      end
+
+      # Attempt to retrieve the title from the meta tag
+      title_element = doc.at_css('title')
+      if title_element && title_element.text
+        data[:title] ||= title_element.text unless Onebox::Helpers.blank?(title_element.text)
+      end
+
+      data
+    end
+
+  end
+end

lib/onebox/preview.rb

@@ -0,0 +1,96 @@
+# frozen_string_literal: true
+
+module Onebox
+  class Preview
+    # see https://bugs.ruby-lang.org/issues/14688
+    client_exception = defined?(Net::HTTPClientException) ? Net::HTTPClientException : Net::HTTPServerException
+    WEB_EXCEPTIONS ||= [client_exception, OpenURI::HTTPError, Timeout::Error, Net::HTTPError, Errno::ECONNREFUSED]
+
+    def initialize(url, options = Onebox.options)
+      @url = url
+      @options = options.dup
+
+      allowed_origins = @options[:allowed_iframe_origins] || Onebox::Engine.all_iframe_origins
+      @options[:allowed_iframe_regexes] = Engine.origins_to_regexes(allowed_origins)
+
+      @engine_class = Matcher.new(@url, @options).oneboxed
+    end
+
+    def to_s
+      return "" unless engine
+      sanitize process_html engine_html
+    rescue *WEB_EXCEPTIONS
+      ""
+    end
+
+    def placeholder_html
+      return "" unless engine
+      sanitize process_html engine.placeholder_html
+    rescue *WEB_EXCEPTIONS
+      ""
+    end
+
+    def errors
+      return {} unless engine
+      engine.errors
+    end
+
+    def data
+      return {} unless engine
+      engine.data
+    end
+
+    def options
+      OpenStruct.new(@options)
+    end
+
+    private
+
+    def engine_html
+      engine.to_html
+    end
+
+    def process_html(html)
+      return "" unless html
+
+      if @options[:max_width]
+        doc = Nokogiri::HTML5::fragment(html)
+        if doc
+          doc.css('[width]').each do |e|
+            width = e['width'].to_i
+
+            if width > @options[:max_width]
+              height = e['height'].to_i
+              if (height > 0)
+                ratio = (height.to_f / width.to_f)
+                e['height'] = (@options[:max_width] * ratio).floor
+              end
+              e['width'] = @options[:max_width]
+            end
+          end
+          return doc.to_html
+        end
+      end
+
+      html
+    end
+
+    def sanitize(html)
+      config = @options[:sanitize_config] || Sanitize::Config::ONEBOX
+      config = config.merge(allowed_iframe_regexes: @options[:allowed_iframe_regexes])
+
+      Sanitize.fragment(html, config)
+    end
+
+    def engine
+      return nil unless @engine_class
+      return @engine if defined?(@engine)
+
+      @engine = @engine_class.new(@url)
+      @engine.options = @options
+      @engine
+    end
+
+    class InvalidURI < StandardError; end
+  end
+end

lib/onebox/sanitize_config.rb

@@ -0,0 +1,72 @@
+# frozen_string_literal: true
+
+class Sanitize
+  module Config
+
+    HTTP_PROTOCOLS ||= ['http', 'https', :relative].freeze
+
+    ONEBOX ||= freeze_config merge(RELAXED,
+      elements: RELAXED[:elements] + %w[audio details embed iframe source video svg path],
+
+      attributes: {
+        'a' => RELAXED[:attributes]['a'] + %w(target),
+        'audio' => %w[controls controlslist],
+        'embed' => %w[height src type width],
+        'iframe' => %w[allowfullscreen frameborder height scrolling src width data-original-href data-unsanitized-src],
+        'source' => %w[src type],
+        'video' => %w[controls height loop width autoplay muted poster controlslist playsinline],
+        'path' => %w[d],
+        'svg' => ['aria-hidden', 'width', 'height', 'viewbox'],
+        'div' => [:data], # any data-* attributes,
+        'span' => [:data], # any data-* attributes
+      },
+
+      add_attributes: {
+        'iframe' => {
+          'seamless' => 'seamless',
+          'sandbox' => 'allow-same-origin allow-scripts allow-forms allow-popups allow-popups-to-escape-sandbox' \
+                       ' allow-presentation',
+        }
+      },
+
+      transformers: (RELAXED[:transformers] || []) + [
+        lambda do |env|
+          next unless env[:node_name] == 'a'
+          a_tag = env[:node]
+          a_tag['href'] ||= '#'
+          if a_tag['href'] =~ %r{^(?:[a-z]+:)?//}
+            a_tag['rel'] = 'nofollow ugc noopener'
+          else
+            a_tag.remove_attribute('target')
+          end
+        end,
+
+        lambda do |env|
+          next unless env[:node_name] == 'iframe'
+
+          iframe = env[:node]
+          allowed_regexes = env[:config][:allowed_iframe_regexes] || [/.*/]
+
+          allowed = allowed_regexes.any? { |r| iframe["src"] =~ r }
+
+          if !allowed
+            # add a data attribute with the blocked src. This is not required
+            # but makes it much easier to troubleshoot onebox issues
+            iframe["data-unsanitized-src"] = iframe["src"]
+            iframe.remove_attribute("src")
+          end
+        end
+      ],
+
+      protocols: {
+        'embed' => { 'src' => HTTP_PROTOCOLS },
+        'iframe' => { 'src' => HTTP_PROTOCOLS },
+        'source' => { 'src' => HTTP_PROTOCOLS },
+      },
+
+      css: {
+        properties: RELAXED[:css][:properties] + %w[--aspect-ratio]
+      }
+    )
+  end
+end

lib/onebox/status_check.rb

@@ -0,0 +1,46 @@
+# frozen_string_literal: true
+
+module Onebox
+  class StatusCheck
+    def initialize(url, options = Onebox.options)
+      @url = url
+      @options = options
+      @status = -1
+    end
+
+    def ok?
+      status > 199 && status < 300
+    end
+
+    def status
+      check if @status == -1
+      @status
+    end
+
+    def human_status
+      case status
+      when 0
+        :connection_error
+      when 200..299
+        :success
+      when 400..499
+        :client_error
+      when 500..599
+        :server_error
+      else
+        :unknown_error
+      end
+    end
+
+    private
+
+    def check
+      res = URI.open(@url, read_timeout: (@options.timeout || Onebox.options.timeout))
+      @status = res.status.first.to_i
+    rescue OpenURI::HTTPError => e
+      @status = e.io.status.first.to_i
+    rescue Timeout::Error, Errno::ECONNREFUSED, Net::HTTPError
+      @status = 0
+    end
+  end
+end

lib/onebox/template_support.rb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+module Onebox
+  module TemplateSupport
+    def load_paths
+      Onebox.options.load_paths.select(&method(:template?))
+    end
+
+    def template?(path)
+      File.exist?(File.join(path, "#{template_name}.#{template_extension}"))
+    end
+  end
+end

lib/onebox/templates/_layout.mustache

@@ -0,0 +1,25 @@
+<aside class="onebox {{subname}}">
+  <header class="source">
+    {{#favicon}}
+      <img src="{{favicon}}" class="site-icon"/>
+    {{/favicon}}
+
+    {{#article_published_time}}
+      <a href="{{link}}" target="_blank" rel="noopener" title="{{article_published_time_title}}">{{domain}} &ndash; {{article_published_time}}</a>
+    {{/article_published_time}}
+    {{^article_published_time}}
+      <a href="{{link}}" target="_blank" rel="noopener">{{domain}}</a>
+    {{/article_published_time}}
+  </header>
+
+  <article class="onebox-body">
+    {{{view}}}
+  </article>
+
+  <div class="onebox-metadata">
+    {{#metadata_1_label}}<span style="float: left;">{{metadata_1_label}}: {{metadata_1_value}}</span>{{/metadata_1_label}}
+    {{#metadata_2_label}}<span style="float: right;">{{metadata_2_label}}: {{metadata_2_value}}</span>{{/metadata_2_label}}
+  </div>
+
+  <div style="clear: both"></div>
+</aside>

lib/onebox/templates/allowlistedgeneric.mustache

@@ -0,0 +1,16 @@
+{{#image}}<img src="{{image}}" class="thumbnail"/>{{/image}}
+
+<h3><a href="{{link}}" target="_blank" rel="noopener">{{title}}</a></h3>
+
+{{#description}}
+  <p>{{description}}</p>
+{{/description}}
+
+{{#data_1}}
+  <p>
+    <span class="label1">{{label_1}}: {{data_1}}</span>
+    {{#data_2}}
+      <span class="label2">{{label_2}}: {{data_2}}</span>
+    {{/data_2}}
+  </p>
+{{/data_1}}

lib/onebox/templates/amazon.mustache

@@ -0,0 +1,15 @@
+{{#image}}<img src="{{image}}" class="thumbnail">{{/image}}
+
+<h3><a href="{{link}}" target="_blank" rel="noopener">{{title}}</a></h3>
+
+{{#by_info}}<b>{{by_info}}</b>{{/by_info}}
+
+<p>{{description}}</p>
+
+<p>
+  {{#rating}}{{rating}}{{/rating}}
+  {{#isbn_asin}}{{isbn_asin_text}}: {{isbn_asin}}, {{/isbn_asin}}
+  {{#publisher}}{{publisher}}, {{/publisher}}
+  {{#published}}{{published}}{{/published}}
+  {{#price}}<strong>{{price}}</strong>{{/price}}
+</p>

lib/onebox/templates/github/github_body.mustache

@@ -0,0 +1,5 @@
+{{#body}}
+  <div class="github-row">
+    <p class="github-body-container">{{body}}{{#excerpt}}<span class="show-more-container"><a href="{{html_url}}" target="_blank" rel="noopener" class="show-more">…</a>{{/excerpt}}</span>{{#excerpt}}<span class="excerpt hidden">{{excerpt}}</span>{{/excerpt}}</p>
+  </div>
+{{/body}}

lib/onebox/templates/githubblob.mustache

@@ -0,0 +1,54 @@
+<h4><a href="{{link}}" target="_blank" rel="noopener">{{title}}</a></h4>
+
+{{^has_lines}}
+  {{#model_file}}
+    <iframe class="render-viewer" width="{{width}}" height="{{height}}" src="{{content}}" sandbox="allow-scripts allow-same-origin allow-top-navigation ">
+      Viewer requires iframe.
+    </iframe>
+  {{/model_file}}
+
+  {{^model_file}}
+    <pre><code class="{{lang}}">{{content}}</code></pre>
+  {{/model_file}}
+{{/has_lines}}
+
+{{#has_lines}}
+  {{! This is a template comment  | Sample rules for this box
+  <style>
+  pre.onebox code ol{
+    margin-left:0px;
+  }
+  pre.onebox code ol .lines{
+    margin-left:30px;
+  }
+  pre.onebox code ol.lines li {
+    list-style-type: decimal;
+    margin-left:45px;
+  }
+  pre.onebox code li{
+    list-style-type: none;
+    background-color:#fff;
+    border-bottom:1px solid #F0F0F0;
+    padding-left:5px;
+
+  }
+  pre.onebox code li.selected{
+    background-color:#cfc
+  }
+  </style>
+  }}
+
+  <pre class="onebox">
+    <code class="{{lang}}">
+      <ol class="start lines" start="{{cr_results.from}}" style="counter-reset: li-counter {{cr_results.from_minus_one}} ;">
+        {{#lines}}
+          <li{{#selected}} class="selected"{{/selected}}>{{data}}</li>
+        {{/lines}}
+      </ol>
+    </code>
+  </pre>
+{{/has_lines}}
+
+{{#truncated}}
+  This file has been truncated. <a href="{{link}}" target="_blank" rel="noopener">show original</a>
+{{/truncated}}

lib/onebox/templates/githubcommit.mustache

@@ -0,0 +1,33 @@
+<div class="github-row">
+  <div class="github-icon-container" title="Commit">
+    <svg width="60" height="60" class="github-icon" viewBox="0 0 14 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M10.86 7c-.45-1.72-2-3-3.86-3-1.86 0-3.41 1.28-3.86 3H0v2h3.14c.45 1.72 2 3 3.86 3 1.86 0 3.41-1.28 3.86-3H14V7h-3.14zM7 10.2c-1.22 0-2.2-.98-2.2-2.2 0-1.22.98-2.2 2.2-2.2 1.22 0 2.2.98 2.2 2.2 0 1.22-.98 2.2-2.2 2.2z"></path></svg>
+  </div>
+
+  <div class="github-info-container">
+    <h4>
+      <a href="{{html_url}}" target="_blank" rel="noopener">{{title}}</a>
+    </h4>
+
+    <div class="github-info">
+      <div class="date">
+        committed <span class="discourse-local-date" data-format="ll" data-date="{{committed_at_date}}" data-time="{{committed_at_time}}" data-timezone="UTC">{{committed_at}}</span>
+      </div>
+
+      <div class="user">
+        <a href="{{author.html_url}}" target="_blank" rel="noopener">
+          <img alt="{{author.login}}" src="{{author.avatar_url}}" class="onebox-avatar-inline" width="20" height="20">
+          {{author.login}}
+        </a>
+      </div>
+
+      <div class="lines" title="changed {{files.length}} files with {{stats.additions}} additions and {{stats.deletions}} deletions">
+        <a href="{{html_url}}" target="_blank" rel="noopener">
+          <span class="added">+{{stats.additions}}</span>
+          <span class="removed">-{{stats.deletions}}</span>
+        </a>
+      </div>
+    </div>
+  </div>
+</div>
+
+{{> github/github_body}}

lib/onebox/templates/githubfolder.mustache

@@ -0,0 +1,11 @@
+{{#image}}<img src="{{image}}" class="thumbnail"/>{{/image}}
+
+<h3><a href="{{link}}" target="_blank" rel="noopener">{{title}}</a></h3>
+
+{{#path}}
+  <p><a href="{{link}}" target="_blank" rel="noopener">{{path}}</a></p>
+{{/path}}
+
+{{#description}}
+  <p><span class="label1">{{description}}</span></p>
+{{/description}}

lib/onebox/templates/githubgist.mustache

@@ -0,0 +1,15 @@
+<h4><a href="{{link}}" target="_blank" rel="noopener">{{link}}</a></h4>
+
+{{#gist_files}}
+  <h5>{{filename}}</h5>
+  <pre><code class="{{language}}">{{content}}</code></pre>
+  {{#truncated?}}
+    This file has been truncated. <a href="{{link}}" target="_blank" rel="noopener">show original</a>
+  {{/truncated?}}
+{{/gist_files}}
+
+<p>
+  {{#truncated_files?}}
+    There are more than three files. <a href="{{link}}" target="_blank" rel="noopener">show original</a>
+  {{/truncated_files?}}
+</p>

lib/onebox/templates/githubissue.mustache

@@ -0,0 +1,38 @@
+<div class="github-row">
+  <div class="github-icon-container" title="Issue">
+	  <svg width="60" height="60" class="github-icon" viewBox="0 0 14 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
+  </div>
+
+  <div class="github-info-container">
+    <h4>
+      <a href="{{link}}" target="_blank" rel="noopener">{{title}}</a>
+    </h4>
+
+    <div class="github-info">
+      <div class="date">
+        opened <span class="discourse-local-date" data-format="ll" data-date="{{created_at_date}}" data-time="{{created_at_time}}" data-timezone="UTC">{{created_at}}</span>
+      </div>
+
+      {{#closed_at}}
+        <div class="date">
+          closed <span class="discourse-local-date" data-format="ll" data-date="{{closed_at_date}}" data-time="{{closed_at_time}}" data-timezone="UTC">{{closed_at}}</span>
+        </div>
+      {{/closed_at}}
+
+      <div class="user">
+        <a href="{{user.html_url}}" target="_blank" rel="noopener">
+          <img alt="{{user.login}}" src="{{user.avatar_url}}" class="onebox-avatar-inline" width="20" height="20">
+          {{user.login}}
+        </a>
+      </div>
+    </div>
+
+    <div class="labels">
+      {{#labels}}
+        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">{{name}}</span>
+      {{/labels}}
+    </div>
+  </div>
+</div>
+
+{{> github/github_body}}

lib/onebox/templates/githubpullrequest.mustache

@@ -0,0 +1,37 @@
+<div class="github-row">
+  <div class="github-icon-container" title="Pull Request">
+    <svg width="60" height="60" class="github-icon" viewBox="0 0 12 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M11 11.28V5c-.03-.78-.34-1.47-.94-2.06C9.46 2.35 8.78 2.03 8 2H7V0L4 3l3 3V4h1c.27.02.48.11.69.31.21.2.3.42.31.69v6.28A1.993 1.993 0 0 0 10 15a1.993 1.993 0 0 0 1-3.72zm-1 2.92c-.66 0-1.2-.55-1.2-1.2 0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2 0 .65-.55 1.2-1.2 1.2zM4 3c0-1.11-.89-2-2-2a1.993 1.993 0 0 0-1 3.72v6.56A1.993 1.993 0 0 0 2 15a1.993 1.993 0 0 0 1-3.72V4.72c.59-.34 1-.98 1-1.72zm-.8 10c0 .66-.55 1.2-1.2 1.2-.65 0-1.2-.55-1.2-1.2 0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2zM2 4.2C1.34 4.2.8 3.65.8 3c0-.65.55-1.2 1.2-1.2.65 0 1.2.55 1.2 1.2 0 .65-.55 1.2-1.2 1.2z"></path></svg>
+  </div>
+
+  <div class="github-info-container">
+    <h4>
+      <a href="{{html_url}}" target="_blank" rel="noopener">{{title}}</a>
+    </h4>
+
+    <div class="branches">
+      <code>{{base.label}}</code> ← <code>{{head.label}}</code>
+    </div>
+
+    <div class="github-info">
+      <div class="date">
+        opened <span class="discourse-local-date" data-format="ll" data-date="{{created_at_date}}" data-time="{{created_at_time}}" data-timezone="UTC">{{created_at}}</span>
+      </div>
+
+      <div class="user">
+        <a href="{{user.html_url}}" target="_blank" rel="noopener">
+          <img alt="{{user.login}}" src="{{user.avatar_url}}" class="onebox-avatar-inline" width="20" height="20">
+          {{user.login}}
+        </a>
+      </div>
+
+      <div class="lines" title="{{commits}} commits changed {{changed_files}} files with {{additions}} additions and {{deletions}} deletions">
+        <a href="{{html_url}}/files" target="_blank" rel="noopener">
+          <span class="added">+{{additions}}</span>
+          <span class="removed">-{{deletions}}</span>
+        </a>
+      </div>
+    </div>
+  </div>
+</div>
+
+{{> github/github_body}}

lib/onebox/templates/gitlabblob.mustache

@@ -0,0 +1,21 @@
+<h4><a href="{{link}}" target="_blank" rel="noopener">{{title}}</a></h4>
+
+{{^has_lines}}
+  <pre><code class="{{lang}}">{{content}}</code></pre>
+{{/has_lines}}
+
+{{#has_lines}}
+  <pre class="onebox">
+    <code class="{{lang}}">
+      <ol class="start lines" start="{{cr_results.from}}" style="counter-reset: li-counter {{cr_results.from_minus_one}} ;">
+        {{#lines}}
+          <li{{#selected}} class="selected"{{/selected}}>{{data}}</li>
+        {{/lines}}
+      </ol>
+    </code>
+  </pre>
+{{/has_lines}}
+
+{{#truncated}}
+  This file has been truncated. <a href="{{link}}" target="_blank" rel="noopener">show original</a>
+{{/truncated}}

lib/onebox/templates/googledocs.mustache

@@ -0,0 +1,5 @@
+<a href="{{link}}" target="_blank" rel="noopener"><span class="googledocs-onebox-logo g-{{type}}-logo"></span></a>
+
+<h3><a href="{{link}}" target="_blank" rel="noopener">{{title}}</a></h3>
+
+<p>{{description}}</p>

lib/onebox/templates/googledrive.mustache

@@ -0,0 +1,9 @@
+{{^image}}
+  <a href="{{link}}" target="_blank" rel="noopener"><span class="googledocs-onebox-logo g-drive-logo"></span></a>
+{{/image}}
+
+{{#image}}<img src="{{image}}" class="thumbnail">{{/image}}
+
+<h3><a href="{{link}}" target="_blank" rel="noopener">{{title}}</a></h3>
+
+<p>{{description}}</p>

lib/onebox/templates/googleplayapp.mustache

@@ -0,0 +1,5 @@
+<h3>{{title}}</h3>
+
+<img src="{{image}}" class="thumbnail">
+<p>{{description}}</p>
+<em>{{price}}</em>

lib/onebox/templates/instagram.mustache

@@ -0,0 +1,13 @@
+<h3><a href="{{{link}}}" target="_blank" rel="noopener">{{title}}</a></h3>
+
+{{#image}}
+  <div class="instagram-images">
+    <a href="{{{link}}}" target="_blank" rel="noopener">
+      <img class="instagram-image" src="{{{image}}}">
+    </a>
+  </div>
+{{/image}}
+
+{{#description}}
+  <div class="instagram-description">{{description}}</div>
+{{/description}}

lib/onebox/templates/pastebin.mustache

@@ -0,0 +1,7 @@
+<h4><a href="{{link}}" target="_blank" rel="noopener">{{link}}</a></h4>
+
+<pre><code class="lang-auto">{{content}}</code></pre>
+
+{{#truncated?}}
+  This paste has been truncated. <a href="{{link}}" target="_blank" rel="noopener">show original</a>
+{{/truncated?}}

lib/onebox/templates/pdf.mustache

@@ -0,0 +1,7 @@
+<a href="{{link}}" target="_blank" rel="noopener"><span class="pdf-onebox-logo"></span></a>
+
+<h3><a href="{{link}}" target="_blank" rel="noopener">{{title}}</a></h3>
+
+{{#filesize}}
+  <p class="filesize">{{filesize}}</p>
+{{/filesize}}

lib/onebox/templates/pubmed.mustache

@@ -0,0 +1,12 @@
+<h4>
+  <a href="{{link}}" target="_blank" rel="noopener">{{title}}</a>
+</h4>
+
+<div class="date">
+  {{authors}},
+  <i>{{journal}}</i>, {{date}}
+</div>
+
+<p class="pubmed-abstract">
+  {{abstract}}
+</p>

lib/onebox/templates/stackexchange.mustache

@@ -0,0 +1,22 @@
+{{#owner.profile_image}}
+  <a href="{{owner.link}}" target="_blank" rel="noopener">
+    <img alt="{{owner.display_name}}" src="{{owner.profile_image}}" class="thumbnail onebox-avatar">
+  </a>
+{{/owner.profile_image}}
+
+<h4>
+  <a href="{{link}}" target="_blank" rel="noopener">{{{title}}}</a>
+</h4>
+
+<div class="tags">
+  <strong>{{tags}}</strong>
+</div>
+
+<div class="date">
+  {{#is_question}}asked by{{/is_question}}
+  {{#is_answer}}answered by{{/is_answer}}
+  <a href="{{owner.link}}" target="_blank" rel="noopener">
+    {{owner.display_name}}
+  </a>
+  on <a href="{{link}}" target="_blank" rel="noopener">{{creation_date}}</a>
+</div>