diff --git a/.rubocop.yml b/.rubocop.yml index e8d9080..1e991b8 100644 --- a/.rubocop.yml +++ b/.rubocop.yml @@ -27,3 +27,6 @@ RSpec/DescribeClass: Exclude: - samples/**/* - spec/integration/**/* +RSpec/MultipleDescribes: + Exclude: + - samples/**/* diff --git a/Rakefile b/Rakefile index fc5bd75..ae15e73 100644 --- a/Rakefile +++ b/Rakefile @@ -37,6 +37,7 @@ desc 'Run the samples' task :samples do FileUtils.rm_rf('samples/tmp') sh 'bundle exec polytrix exec --code2doc samples/*.rb samples/*.sh' + sh 'bundle exec polytrix exec --code2doc samples/*.json --lang js' end desc 'Build the documentation from the samples' diff --git a/docs/actors.md b/docs/actors.md new file mode 100644 index 0000000..8638575 --- /dev/null +++ b/docs/actors.md @@ -0,0 +1,58 @@ +Pacto uses actors stub **providers** or simulate **consumers** based on the contract. There +are two built-in actors. The FromExamples actor will produce requests or responses based on +the examples in the contract. The JSONGenerator actor will attempt to generate requests or +responses that match the JSON schema, though it only works for simple schemas. The FromExamples +actor is the default, but falls back to the JSONGenerator actor if there are no examples available. +Consider the following contract: + +```json +{ + "name": "Ping", + "request": { + "headers": { + }, + "http_method": "get", + "path": "/api/ping" + }, + "response": { + "headers": { + "Content-Type": "application/json" + }, + "status": 200, + "schema": { + "$schema": "http://json-schema.org/draft-03/schema#", + "type": "object", + "required": true, + "properties": { + "ping": { + "type": "string", + "required": true + } + } + } + }, + "examples": { + "default": { + "request": { + }, + "response": { + "body": { + "ping": "pong - from the example!" + } + } + } + } +} +``` +Then Pacto will generate the following response by default (via FromExamples): + +```json +{"ping":"pong - from the example!"} +``` + +If you didn't have an example, then Pacto generate very basic mock data based on the schema types, +producing something like: + +```json +{"ping":"bar"} +``` diff --git a/docs/clerks.md b/docs/clerks.md new file mode 100644 index 0000000..20a93f7 --- /dev/null +++ b/docs/clerks.md @@ -0,0 +1,40 @@ +Pacto clerks are responsible for loading contracts. Pacto has built-in support for a native +contract format, but we've setup a Clerks plugin API so you can more easily load information +from other formats, like [Swagger](https://github.com/wordnik/swagger-spec), +[apiblueprint](http://apiblueprint.org/), or [RAML](http://raml.org/). +Note: This is a preliminary API and may change in the future, including adding support +for conversion between formats, or generating each format from real HTTP interactions. +In order to add a loading clerk, you just implement a class that responds to build_from_file +and returns Contract object or a collection of Contract objects. + +```rb +require 'yaml' +require 'pacto' + +class SimpleYAMLClerk + def build_from_file(path, _host) + data = YAML.load(File.read(path)) + data['services'].map do | service_name, service_definition | + request_clause = Pacto::RequestClause.new service_definition['request'] + response_clause = Pacto::ResponseClause.new service_definition['response'] + Pacto::Contract.new(name: service_name, request: request_clause, response: response_clause) + end + end +end +``` + +You can then register the clerk with Pacto: + +```rb +Pacto.contract_factory.add_factory :simple_yaml, SimpleYAMLClerk.new +``` + +And then you can use it with the normal clerks API, by passing the identifier you used to register +the clerk: + +```rb +contracts = Pacto.load_contracts 'simple_service_map.yaml', 'http://example.com', :simple_yaml +contract_names = contracts.map(&:name) +puts "Defined contracts: #{contract_names}" +``` + diff --git a/docs/configuration.md b/docs/configuration.md new file mode 100644 index 0000000..5e60a3a --- /dev/null +++ b/docs/configuration.md @@ -0,0 +1,39 @@ +Pacto will disable live connections, so you will get an error if +your code unexpectedly calls an service that was not stubbed. If you +want to re-enable connections, run `WebMock.allow_net_connect!` after +requiring pacto. + +```rb +require 'pacto' +WebMock.allow_net_connect! +``` + +Pacto can be configured via a block: + +```rb +Pacto.configure do |c| + c.contracts_path = 'contracts' # Path for loading/storing contracts. + c.strict_matchers = true # If the request matching should be strict (especially regarding HTTP Headers). + c.stenographer_log_file = nil # Set to nil to disable the stenographer log. +end +``` + +You can also do inline configuration. This example tells the +[json-schema-generator](https://github.com/maxlinc/json-schema-generator) to +store default values in the schema. + +```rb +Pacto.configuration.generator_options = { defaults: true } +``` + +All Pacto configuration and metrics can be reset ia `Pacto.clear!`. If you're using +RSpec you may want to clear between each scenario: +If you're using Pacto's rspec matchers you might want to configure a reset between each scenario + +```rb +require 'pacto/rspec' +RSpec.configure do |c| + c.after(:each) { Pacto.clear! } +end +``` + diff --git a/docs/contracts.md b/docs/contracts.md new file mode 100644 index 0000000..7843ce5 --- /dev/null +++ b/docs/contracts.md @@ -0,0 +1,109 @@ +Pacto Contracts describe the constraints we want to put on interactions between a consumer and a provider. It sets some expectations about the headers expected for both the request and response, the expected response status code. It also uses [json-schema](http://json-schema.org/) to define the allowable request body (if one should exist) and response body. + +```js +{ +``` + +The Request section comes first. In this case, we're just describing a simple get request that does not require any parameters or a request body. + +```js + "request": { + "headers": { +``` + +A request must exactly match these headers for Pacto to believe the request matches the contract, unless `Pacto.configuration.strict_matchers` is false. + +```js + "Accept": "application/vnd.github.beta+json", + "Accept-Encoding": "gzip;q=1.0,deflate;q=0.6,identity;q=0.3" + }, +``` + +The `method` and `path` are required. The `path` may be an [rfc6570 URI template](http://tools.ietf.org/html/rfc6570) for more flexible matching. + +```js + "http_method": "get", + "path": "/repos/thoughtworks/pacto/readme" + }, + "response": { + "headers": { + "Content-Type": "application/json; charset=utf-8", + "Status": "200 OK", + "Cache-Control": "public, max-age=60, s-maxage=60", + "Etag": "\"fc8e78b0a9694de66d47317768b20820\"", + "Vary": "Accept, Accept-Encoding", + "Access-Control-Allow-Credentials": "true", + "Access-Control-Expose-Headers": "ETag, Link, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval", + "Access-Control-Allow-Origin": "*" + }, + "status": 200, + "schema": { + "$schema": "http://json-schema.org/draft-03/schema#", + "description": "Generated from https://api.github.com/repos/thoughtworks/pacto/readme with shasum 3ae59164c6d9f84c0a81f21fb63e17b3b8ce6894", + "type": "object", + "required": true, + "properties": { + "name": { + "type": "string", + "required": true + }, + "path": { + "type": "string", + "required": true + }, + "sha": { + "type": "string", + "required": true + }, + "size": { + "type": "integer", + "required": true + }, + "url": { + "type": "string", + "required": true + }, + "html_url": { + "type": "string", + "required": true + }, + "git_url": { + "type": "string", + "required": true + }, + "type": { + "type": "string", + "required": true + }, + "content": { + "type": "string", + "required": true + }, + "encoding": { + "type": "string", + "required": true + }, + "_links": { + "type": "object", + "required": true, + "properties": { + "self": { + "type": "string", + "required": true + }, + "git": { + "type": "string", + "required": true + }, + "html": { + "type": "string", + "required": true + } + } + } + } + } + } +} +``` + diff --git a/docs/cops.md b/docs/cops.md new file mode 100644 index 0000000..2e022ad --- /dev/null +++ b/docs/cops.md @@ -0,0 +1,50 @@ +You can create a custom cop that investigates the request/response and sees if it complies with a +contract. The cop should return a list of citations if it finds any problems. + +```rb +require 'pacto' +class MyCustomCop + def investigate(_request, _response, contract) + citations = [] + citations << 'Contract must have a request schema' if contract.request.schema.empty? + citations << 'Contract must have a response schema' if contract.response.schema.empty? + citations + end +end +``` + +You can activate the cop by adding it to the active_cops. The active_cops are reset +by `Pacto.clear!` + +```rb +Pacto::Cops.active_cops << MyCustomCop.new +``` + +Or you could add it as a registered cop. These cops are not cleared - they form the +default set of Cops used by Pacto: + +```rb +Pacto::Cops.register_cop MyCustomCop.new +``` + +The cops will be used to validate any service requests/responses detected by Pacto, +including when we simulate consumers: + +```rb +Pacto.validate! +contracts = Pacto.load_contracts('contracts', 'http://localhost:5000') +contracts.stub_providers +puts contracts.simulate_consumers +``` + +You could also completely reset the registered cops if you don't want to use +all of Pacto's built-in cops: + +```rb +Pacto::Cops.registered_cops.clear +Pacto::Cops.register_cop Pacto::Cops::ResponseBodyCop + +contracts = Pacto.load_contracts('contracts', 'http://localhost:5000') +puts contracts.simulate_consumers +``` + diff --git a/docs/forensics.md b/docs/forensics.md new file mode 100644 index 0000000..b0267fe --- /dev/null +++ b/docs/forensics.md @@ -0,0 +1,64 @@ +Pacto has a few RSpec matchers to help you ensure a **consumer** and **producer** are +interacting properly. First, let's setup the rspec suite. + +```rb +require 'rspec/autorun' # Not generally needed +require 'pacto/rspec' +WebMock.allow_net_connect! +Pacto.validate! +Pacto.load_contracts('contracts', 'http://localhost:5000').stub_providers +``` + +It's usually a good idea to reset Pacto between each scenario. `Pacto.reset` just clears the +data and metrics about which services were called. `Pacto.clear!` also resets all configuration +and plugins. + +```rb +RSpec.configure do |c| + c.after(:each) { Pacto.reset } +end +``` + +Pacto provides some RSpec matchers related to contract testing, like making sure +Pacto didn't received any unrecognized requests (`have_unmatched_requests`) and that +the HTTP requests matched up with the terms of the contract (`have_failed_investigations`). + +```rb +describe Faraday do + let(:connection) { described_class.new(url: 'http://localhost:5000') } + + it 'passes contract tests' do + connection.get '/api/ping' + expect(Pacto).to_not have_failed_investigations + expect(Pacto).to_not have_unmatched_requests + end +end +``` + +There are also some matchers for collaboration testing, so you can make sure each scenario is +calling the expected services and sending the right type of data. + +```rb +describe Faraday do + let(:connection) { described_class.new(url: 'http://localhost:5000') } + before(:each) do + connection.get '/api/ping' + + connection.post do |req| + req.url '/api/echo' + req.headers['Content-Type'] = 'application/json' + req.body = '{"foo": "bar"}' + end + end + + it 'calls the ping service' do + expect(Pacto).to have_validated(:get, 'http://localhost:5000/api/ping').against_contract('Ping') + end + + it 'sends data to the echo service' do + expect(Pacto).to have_investigated('Echo').with_request(body: hash_including('foo' => 'bar')) + expect(Pacto).to have_investigated('Echo').with_response(body: hash_including('foo' => 'bar')) + end +end +``` + diff --git a/docs/generation.md b/docs/generation.md new file mode 100644 index 0000000..7ec7ab4 --- /dev/null +++ b/docs/generation.md @@ -0,0 +1,48 @@ +Some generation related [configuration](configuration.rb). + +```rb +require 'pacto' +WebMock.allow_net_connect! +Pacto.configure do |c| + c.contracts_path = 'contracts' +end +WebMock.allow_net_connect! +``` + +Once we call `Pacto.generate!`, Pacto will record contracts for all requests it detects. + +```rb +Pacto.generate! +``` + +Now, if we run any code that makes an HTTP call (using an +[HTTP library supported by WebMock](https://github.com/bblimke/webmock#supported-http-libraries)) +then Pacto will generate a Contract based on the HTTP request/response. + +This code snippet will generate a Contract and save it to `contracts/samples/contracts/localhost/api/ping.json`. + +```rb +require 'faraday' +conn = Faraday.new(url: 'http://localhost:5000') +response = conn.get '/api/ping' +``` + +We're getting back real data from GitHub, so this should be the actual file encoding. + +```rb +puts response.body +``` + +The generated contract will contain expectations based on the request/response we observed, +including a best-guess at an appropriate json-schema. Our heuristics certainly aren't foolproof, +so you might want to customize schema! +Here's another sample that sends a post request. + +```rb +conn.post do |req| + req.url '/api/echo' + req.headers['Content-Type'] = 'application/json' + req.body = '{"red fish": "blue fish"}' +end +``` + diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..313570f --- /dev/null +++ b/docs/index.md @@ -0,0 +1,146 @@ +[](http://badge.fury.io/rb/pacto) +[](https://travis-ci.org/thoughtworks/pacto) +[](https://codeclimate.com/github/thoughtworks/pacto) +[](https://gemnasium.com/thoughtworks/pacto) +[](https://coveralls.io/r/thoughtworks/pacto) + +**If you're viewing this at https://github.com/thoughtworks/pacto, +you're reading the documentation for the master branch. +[View documentation for the latest release +(0.3.0).](https://github.com/thoughtworks/pacto/tree/v0.3.0)** + +# Pacto +## Who is Pacto? + +Pacto is a judge that arbitrates contract disputes between a **service provider** and one or more **consumers**. It is a framework for [Integration Contract Testing](http://martinfowler.com/bliki/IntegrationContractTest.html), and service evolution patterns like [Consumer-Driven Contracts](http://thoughtworks.github.io/pacto/patterns/cdc/) or [Documentation-Driven Contracts](http://thoughtworks.github.io/pacto/patterns/documentation_driven/). + +## The litigants + +Pacto helps settle disputes between **service providers** and **service consumers** of RESTful JSON services. The **provider** is the one that implements the service, which may be used by multiple **consumers**. This is done by [Integration Contract Testing](http://martinfowler.com/bliki/IntegrationContractTest.html), where the contract stays the same but the provider changes. + +## Litigators + +Someone needs to accuse the **providers** or **consumers** of wrongdoing! Pacto integrates with a few different test frameworks to give you options: + +- Pacto easily integrates with [RSpec](http://rspec.info/), including some [custom matchers](#forensics). +- Pacto provides some [simple rake tasks](rake_tasks.md) to run some basic tests from the command line. +- If you're testing non-Ruby projects, you can use the [Pacto Server](server.md) as a proxy to intercept and validate requests. You can also use it in conjunction with [Polytrix](https://github.com/rackerlabs/polytrix). + +## Contracts + +Pacto considers two major terms in order decide if there has been a breach of contract: the **request clause** and the **response clause**. + +The **request clause** defines what information must be sent by the **consumer** to the **provider** in order to compel them to render a service. The request clause often describes the required HTTP request headers like `Content-Type`, the required parameters, and the required request body (defined in [json-schema](http://json-schema.org/)) when applicable. Providers are not held liable for failing to deliver services for invalid requests. + +The **response clause** defines what information must be returned by the **provider** to the **consumer** in order to successfully complete the transaction. This commonly includes HTTP response headers like `Location` as well as the required response body (also defined in [json-schema](http://json-schema.org/)). + +See the [Contracts documentation](contracts.md) for more details. + + + +## Enforcement + +### Cops +**Cops** help Pacto investigate interactions between **consumers** and **providers** and determine if either has violated the contract. + +Pacto has a few built-in cops that are on-duty by default. These cops will: +- Ensure the request body matches the contract requirements (if a request body is needed) +- Ensure the response headers match the contract requirements +- Ensure the response HTTP status matches the contract requirements +- Ensure the response body matches the contract requirements + +### Forensics + +Sometimes it looks like you're following a contract, but digital forensics reveals there's some fraud going on. Pacto provides RSpec matchers to help you catch these patterns that let you do the [collaboration tests](http://programmers.stackexchange.com/questions/135011/removing-the-integration-test-scam-understanding-collaboration-and-contract) that integration contract testing alone would not catch. + +See the [forensics documetation](forensics.md) for more details. + +### Sting operations + +**Note: this is a preview of an upcoming feature. It is not currently available.** + +Pacto **cops** merely observe the interactions between a **consumer** and **provider** and look for problems. A Pacto **sting operation** will alter the interaction in order to try an find problems. + +For example, HTTP header field names are supposed to be case-insensitive [according to RFC 2616 - Hypertext Transfer Protocol -- HTTP/1.1](http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2), but many implementations are tightly coupled to a certain server or implementation and assume header field names have a certain case, like "Content-Type" and not "content-type". Pacto can change the alter the character case of the field names in order to catch consumers or providers that are not following this part of the RFC. + +Another possible sting operation is to introduce network lag, dropped connections, simulate HTTP rate limiting errors, or other issues that a robust consumer are expected to handle. + +You can also add your own custom cops to extend Pacto's abilities or to ensure services are standards that are specific to your organization. See the [Cops documentation](cops.md) for more details. + +## Inside the courtroom + +### Actors + +It's not always practical to test using a **real consumer** and or a **real provider**. Pacto can both **stub providers** and **simulate consumers** so you can test their counterpart in isolation. This makes [Consumer-Driven Contracts](http://thoughtworks.github.io/pacto/patterns/cdc/) easier. You can start testing a consumer against a stubbed provider before the real provider is available, and then hand your contracts and tests over to the team that is going to implement the provider so they can ensure it matches your assumptions. + +See the [Actors documentation](actors.md) for more details. + +### The courtroom reporter + +**Note: this is a preview of an upcoming feature. It is not currently available.** + +Pacto can keep track of which services have been called. If you're a consumer with contracts for a set of services you consumer, this helps you figure out "HTTP Service Coverage", which is quite different from code coverage, so you can see if there's any services you forgot to test, or contracts you're still registering with Pacto even though you no longer use those services. + +If you are a provider that is being used by multiple consumers, you could merge coverage reports from each of them to see which services are being used by each consumer. Once this feature is available, you should be able to create reports that look something like this: + + + +### The Stenographer + +The stenographer keeps a short-hand record of everything that happens in the courtroom. These logs can be used for troubleshooting, creating reports, or re-enacting the courtroom activities at a later date (and with different [actors](#actors). + +The stenographer's logs are similar to HTTP access logs in this respect, but in a format that's more compact and suited to Pacto. A typical HTTP access log might look like this: + +``` +#Fields: date time c-ip cs-username s-ip s-port cs-method cs-uri-stem cs-uri-query sc-status cs(User-Agent) +2014-07-01 17:42:15 127.0.0.1 - 127.0.0.1 80 PUT /store/album/123/cover_art - 201 curl/7.30.0 +2014-07-01 17:42:18 127.0.0.1 - 127.0.0.1 80 GET /store/album/123/cover_art size=small 200 curl/7.30.0 +``` + +If Pacto has the following services registered: + + +Then it can match those requests to request and generate a stenographer log that looks like this: +```ruby +request 'Upload Album Art', values: {album_id: '123'}, response: {status: 201} # no contract violations +request 'Download Album Art', values: {album_id: '123', size: 'small'}, response: {status: 200} # no contract violations +``` + +This log file is designed so Pacto it can be used by Pacto to simulate the requests: +```ruby +Pacto.simulate_consumer :my_client do + request 'Upload Album Art', values: {album_id: '123'}, response: {status: 201} # no contract violations + request 'Download Album Art', values: {album_id: '123', size: 'small'}, response: {status: 200} # no contract violations +end +``` + +Since Pacto has added a layer of abstraction you can experiment with changes to the contracts (including routing) without needing to re-record the interactions with the stenographer. For example Pacto will adjust if you change the route from: + +to + + +### Clerks + +Clerks help Pacto with paperwork. Reading and writing legalese is hard. Pacto clerks help create and load contracts. Currently clerks are responsible for: + +- Generating contracts from real HTTP requests +- Basic support for loading from custom formats + +In the future, we plan for clerks to provide more complete support for: +- Converting from or loading other similar contract formats (e.g. [Swagger](https://github.com/wordnik/swagger-spec), [apiblueprint](http://apiblueprint.org/), or [RAML](http://raml.org/). +- Upgrading contracts from older Pacto versions + +See the [contract generation](generation.md) and [clerks](clerks.md) documentation for more info. + +# Implied Terms + +- Pacto only arbitrates contracts for JSON services. +- Pacto requires Ruby 1.9.3 or newer, though you can use [Polyglot Testing](http://thoughtworks.github.io/pacto/patterns/polyglot/) techniques to support older Rubies and non-Ruby projects. + +# Roadmap + +See the [Pacto Roadmap](https://github.com/thoughtworks/pacto/wiki/Pacto-Roadmap) + +# Contributing + +See [CONTRIBUTING.md](https://github.com/thoughtworks/pacto/blob/master/CONTRIBUTING.md) diff --git a/docs/rake_tasks.md b/docs/rake_tasks.md new file mode 100644 index 0000000..7c6bedf --- /dev/null +++ b/docs/rake_tasks.md @@ -0,0 +1,10 @@ +# Rake tasks +## This is a test! +[That](www.google.com) markdown works + +```sh +bundle exec rake pacto:meta_validate['contracts'] + +bundle exec rake pacto:validate['http://localhost:5000','contracts'] +``` + diff --git a/docs/samples.md b/docs/samples.md new file mode 100644 index 0000000..2bee87e --- /dev/null +++ b/docs/samples.md @@ -0,0 +1,137 @@ +# Overview +Welcome to the Pacto usage samples! +This document gives a quick overview of the main features. + +You can browse the Table of Contents (upper right corner) to view additional samples. + +In addition to this document, here are some highlighted samples: +