Resolve conflicts

Author: ninjascant

README.md

@@ -91,7 +91,7 @@ For the latest version, check out the repo and call
 
 ## Running in Docker
 
-1. Install Docker: https://docs.docker.com/install/
+1. Install Docker: https://docs.docker.com/get-docker/
 
 2. Build a docker image
 

docs/dockerhub.md

@@ -1,7 +1,7 @@
 # Uploading to Docker Hub
 
 ```bash
-ETHEREUMETL_VERSION=1.10.1
+ETHEREUMETL_VERSION=1.11.0
 docker build -t ethereum-etl:${ETHEREUMETL_VERSION} -f Dockerfile .
 docker tag ethereum-etl:${ETHEREUMETL_VERSION} blockchainetl/ethereum-etl:${ETHEREUMETL_VERSION}
 docker push blockchainetl/ethereum-etl:${ETHEREUMETL_VERSION}

docs/exporting-the-blockchain.md

@@ -2,20 +2,20 @@
 
 If you'd like to have blockchain data set up and hosted for you, [get in touch with us at D5](https://d5.ai/?ref=ethereumetl).
 
-1. Install python 3.5.3+ https://www.python.org/downloads/
+1. Install python 3.5.3+: [https://www.python.org/downloads/](https://www.python.org/downloads/)
 
 1. You can use Infura if you don't need ERC20 transfers (Infura doesn't support eth_getFilterLogs JSON RPC method).
 For that use `-p https://mainnet.infura.io` option for the commands below. If you need ERC20 transfers or want to
 export the data ~40 times faster, you will need to set up a local Ethereum node:
 
-1. Install geth https://github.com/ethereum/go-ethereum/wiki/Installing-Geth
+1. Install geth: [https://github.com/ethereum/go-ethereum/wiki/Installing-Geth](https://github.com/ethereum/go-ethereum/wiki/Installing-Geth)
 
 1. Start geth.
 Make sure it downloaded the blocks that you need by executing `eth.syncing` in the JS console.
 You can export blocks below `currentBlock`,
 there is no need to wait until the full sync as the state is not needed (unless you also need contracts bytecode
 and token details; for those you need to wait until the full sync). Note that you may need to wait for another day or 
-   two for the node to download the states. See this issue https://github.com/blockchain-etl/ethereum-etl/issues/265#issuecomment-970451522. 
+   two for the node to download the states. See this issue [https://github.com/blockchain-etl/ethereum-etl/issues/265#issuecomment-970451522](https://github.com/blockchain-etl/ethereum-etl/issues/265#issuecomment-970451522). 
 Make sure to set `--txlookuplimit 0` if you use geth.
 
 1. Install Ethereum ETL: `> pip3 install ethereum-etl`
@@ -42,7 +42,7 @@ output/token_transfers/start_block=00000000/end_block=00099999/token_transfers_0
 
 Should work with geth and parity, on Linux, Mac, Windows.
 If you use Parity you should disable warp mode with `--no-warp` option because warp mode
-does not place all of the block or receipt data into the database https://wiki.parity.io/Getting-Synced
+does not place all of the block or receipt data into the database [https://wiki.parity.io/Getting-Synced](https://wiki.parity.io/Getting-Synced)
 
 If you see weird behavior, e.g. wrong number of rows in the CSV files or corrupted files,
 check out this issue: https://github.com/medvedev1088/ethereum-etl/issues/28

docs/google-bigquery.md

@@ -16,4 +16,4 @@ Read [this article](https://medium.com/google-cloud/building-token-recommender-i
 
 ### Awesome BigQuery Views
 
-https://github.com/blockchain-etl/awesome-bigquery-views
+[https://github.com/blockchain-etl/awesome-bigquery-views](https://github.com/blockchain-etl/awesome-bigquery-views)

docs/index.md

@@ -2,7 +2,7 @@
 
 Ethereum ETL lets you convert blockchain data into convenient formats like CSVs and relational databases.
 
-With 700+ likes on Github, Ethereum ETL is the most popular open source project for Ethereum data.
+With 1,700+ likes on GitHub, Ethereum ETL is the most popular open source project for Ethereum data.
 
 Data is available for you to query right away in [Google BigQuery](https://goo.gl/oY5BCQ).
 

ethereumetl/cli/__init__.py

@@ -48,7 +48,7 @@
 
 
 @click.group()
-@click.version_option(version='1.10.1')
+@click.version_option(version='1.11.0')
 @click.pass_context
 def cli(ctx):
     pass

ethereumetl/cli/stream.py

@@ -58,7 +58,6 @@ def stream(last_synced_block_file, lag, provider_uri, output, start_block, entit
     configure_logging(log_file)
     configure_signals()
     entity_types = parse_entity_types(entity_types)
-    validate_entity_types(entity_types, output)
 
     from ethereumetl.streaming.eth_streamer_adapter import EthStreamerAdapter
     from blockchainetl.streaming.streamer import Streamer
@@ -99,12 +98,6 @@ def parse_entity_types(entity_types):
     return entity_types
 
 
-def validate_entity_types(entity_types, output):
-    if output is not None and 'postgres' in output \
-            and (EntityType.CONTRACT in entity_types or EntityType.TOKEN in entity_types):
-        raise ValueError('contract and token are not yet supported entity types for postgres item exporter.')
-
-
 def pick_random_provider_uri(provider_uri):
     provider_uris = [uri.strip() for uri in provider_uri.split(',')]
     return random.choice(provider_uris)

ethereumetl/streaming/item_exporter_creator.py

@@ -57,7 +57,7 @@ def create_item_exporter(output):
         from blockchainetl.jobs.exporters.converters.unix_timestamp_item_converter import UnixTimestampItemConverter
         from blockchainetl.jobs.exporters.converters.int_to_decimal_item_converter import IntToDecimalItemConverter
         from blockchainetl.jobs.exporters.converters.list_field_item_converter import ListFieldItemConverter
-        from ethereumetl.streaming.postgres_tables import BLOCKS, TRANSACTIONS, LOGS, TOKEN_TRANSFERS, TRACES
+        from ethereumetl.streaming.postgres_tables import BLOCKS, TRANSACTIONS, LOGS, TOKEN_TRANSFERS, TRACES, TOKENS, CONTRACTS
 
         item_exporter = PostgresItemExporter(
             output, item_type_to_insert_stmt_mapping={
@@ -66,6 +66,8 @@ def create_item_exporter(output):
                 'log': create_insert_statement_for_table(LOGS),
                 'token_transfer': create_insert_statement_for_table(TOKEN_TRANSFERS),
                 'trace': create_insert_statement_for_table(TRACES),
+                'token': create_insert_statement_for_table(TOKENS),
+                'contract': create_insert_statement_for_table(CONTRACTS),
             },
             converters=[UnixTimestampItemConverter(), IntToDecimalItemConverter(),
                         ListFieldItemConverter('topics', 'topic', fill=4)])

ethereumetl/streaming/postgres_tables.py

@@ -20,7 +20,9 @@
 #  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 #  SOFTWARE.
 
-from sqlalchemy import Table, Column, Integer, BigInteger, String, Numeric, MetaData, TIMESTAMP
+from sqlalchemy import Table, Column, Integer, BigInteger, Boolean, String, Numeric, \
+    MetaData, PrimaryKeyConstraint, VARCHAR, TIMESTAMP
+from sqlalchemy.dialects.postgresql import ARRAY
 
 metadata = MetaData()
 
@@ -127,4 +129,25 @@
     Column('trace_id', String, primary_key=True),
 )
 
+TOKENS = Table(
+    'tokens', metadata,
+    Column('address', VARCHAR(42)),
+    Column('name', String),
+    Column('symbol', String),
+    Column('decimals', Integer),
+    Column('function_sighashes', ARRAY(String)),
+    Column('total_supply', Numeric(78)),
+    Column('block_number', BigInteger),
+    PrimaryKeyConstraint('address', 'block_number', name='tokens_pk'),
+)
 
+CONTRACTS = Table(
+    'contracts', metadata,
+    Column('address', VARCHAR(42)),
+    Column('bytecode', String),
+    Column('function_sighashes', ARRAY(String)),
+    Column('is_erc20', Boolean),
+    Column('is_erc721', Boolean),
+    Column('block_number', BigInteger),
+    PrimaryKeyConstraint('address', 'block_number', name='contracts_pk'),
+)

mkdocs.yml

@@ -16,3 +16,4 @@ nav:
     - Citing: citing.md
 theme: readthedocs
 repo_url: https://github.com/blockchain-etl/ethereum-etl/
+edit_uri: edit/develop/docs