dcsimg

Importer le journal pour Paleobiology Database

  • Started: February 25, 2022 07:56
  • Completed: 01:45:35.000
  • Failed: No.
  • Status: completed

Events (most recent first):

  • 13:43:14.000 (starts) Running
  • 13:38:10.000 (ends) Complete
  • 13:38:10.000 (ends) TOTAL TIME: 13.2m
  • 13:25:16.000 (infos) removing traits specified in diff file
  • 13:25:14.000 (infos) Connecting to https://content.eol.org/ ...
  • 13:24:59.000 (infos) Connecting to https://content.eol.org/ ...
  • 13:24:59.000 (infos) polling for trait diff metadata: /resources/532/publish_diffs.json?since=1645622839
  • 13:24:59.000 (starts) #publish_traits = TraitBank::Slurp.load_resource_from_repo
  • 13:24:59.000 (starts) Running
  • 17:56:21.000 (ends) Complete
  • 17:56:21.000 (ends) TOTAL TIME: 10.0h
  • 17:56:19.000 (starts) TraitBank::Denormalizer.update_resource_vernaculars
  • 17:56:19.000 (starts) Running
  • 17:56:13.000 (starts) Resource#fix_native_nodes
  • 17:56:13.000 (starts) Running
  • 17:56:13.000 (warns) Trait Publishing failed: execution expired (Delayed::Worker.max_run_time is only 36000 seconds) FROM /app/lib/trait_bank/slurp.rb:366:in `sleep' << /app/lib/trait_bank/slurp.rb:366:in `block in break_up_large_file' << /app/lib/trait_bank/slurp.rb:357:in `break_up_large_file' << /app/lib/trait_bank/slurp.rb:343:in `break_up_large_files' << /app/lib/trait_bank/slurp.rb:289:in `load_csv' << /app/lib/trait_bank/slurp.rb:612:in `add_traits'
  • 17:07:38.000 (infos) Waiting 4200 seconds for the last 'chunk' to be added to neo4j...
  • 17:06:19.000 (warns) FAILED on build_nodes query (Trait), will re-try once...
  • 17:06:19.000 (warns) The transaction has been terminated. Retry your operation in a new transaction, and you should see a successful result. The transaction has not completed within the specified timeout (dbms.transaction.timeout). You may want to retry with a longer timeout.
  • 17:06:19.000 (warns) Exception (Neo4j::Driver::Exceptions::ClientException) QUERY: {USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'https://eol.org/data/paleobiology_db/new_traits_1645545266_1645721181_chunk_13.csv' AS row WITH row WHERE 1=1 MERGE (trait:Trait { eol_pk: row.eol_pk }) ON CREATE SET trait.resource_pk = row.resource_pk ON MATCH SET trait.resource_pk = row.resource_pk ON CREATE SET trait.source = row.source ON MATCH SET trait.source = row.source ON CREATE SET trait.literal = row.literal ON MATCH SET trait.literal = row.literal ON CREATE SET trait.measurement = row.measurement ON MATCH SET trait.measurement = row.measurement ON CREATE SET trait.scientific_name = row.scientific_name ON MATCH SET trait.scientific_name = row.scientific_name ON CREATE SET trait.normal_measurement = row.normal_measurement ON MATCH SET trait.normal_measurement = row.normal_measurement ON CREATE SET trait.sample_size = row.sample_size ON MATCH SET trait.sample_size = row.sample_size ON CREATE SET trait.citation = row.citation ON MATCH SET trait.citation = row.citation ON CREATE SET trait.remarks = row.remarks ON MATCH SET trait.remarks = row.remarks ON CREATE SET trait.method = row.method ON MATCH SET trait.method = row.method} MESSAGE: The transaction has been terminated. Retry your operation in a new transaction, and you should see a successful result. The transaction has not completed within the specified timeout (dbms.transaction.timeout). You may want to retry with a longer timeout.
  • 17:04:26.000 (infos) Importing 128000 rows from new_traits_1645545266_1645721181_chunk_13.csv
  • 15:59:25.000 (infos) Waiting 3900 seconds for the last 'chunk' to be added to neo4j...
  • 15:57:58.000 (infos) Importing 128000 rows from new_traits_1645545266_1645721181_chunk_12.csv
  • 14:57:57.000 (infos) Waiting 3600 seconds for the last 'chunk' to be added to neo4j...
  • 14:56:28.000 (infos) Importing 128000 rows from new_traits_1645545266_1645721181_chunk_11.csv
  • 14:01:26.000 (infos) Waiting 3300 seconds for the last 'chunk' to be added to neo4j...
  • 13:59:56.000 (infos) Importing 128000 rows from new_traits_1645545266_1645721181_chunk_10.csv
  • 13:09:55.000 (infos) Waiting 3000 seconds for the last 'chunk' to be added to neo4j...
  • 13:08:33.000 (warns) FAILED on build_nodes query (Trait), will re-try once...
  • 13:08:33.000 (warns) The transaction has been terminated. Retry your operation in a new transaction, and you should see a successful result. The transaction has not completed within the specified timeout (dbms.transaction.timeout). You may want to retry with a longer timeout.
  • 13:08:33.000 (warns) Exception (Neo4j::Driver::Exceptions::ClientException) QUERY: {USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'https://eol.org/data/paleobiology_db/new_traits_1645545266_1645721181_chunk_9.csv' AS row WITH row WHERE 1=1 MERGE (trait:Trait { eol_pk: row.eol_pk }) ON CREATE SET trait.resource_pk = row.resource_pk ON MATCH SET trait.resource_pk = row.resource_pk ON CREATE SET trait.source = row.source ON MATCH SET trait.source = row.source ON CREATE SET trait.literal = row.literal ON MATCH SET trait.literal = row.literal ON CREATE SET trait.measurement = row.measurement ON MATCH SET trait.measurement = row.measurement ON CREATE SET trait.scientific_name = row.scientific_name ON MATCH SET trait.scientific_name = row.scientific_name ON CREATE SET trait.normal_measurement = row.normal_measurement ON MATCH SET trait.normal_measurement = row.normal_measurement ON CREATE SET trait.sample_size = row.sample_size ON MATCH SET trait.sample_size = row.sample_size ON CREATE SET trait.citation = row.citation ON MATCH SET trait.citation = row.citation ON CREATE SET trait.remarks = row.remarks ON MATCH SET trait.remarks = row.remarks ON CREATE SET trait.method = row.method ON MATCH SET trait.method = row.method} MESSAGE: The transaction has been terminated. Retry your operation in a new transaction, and you should see a successful result. The transaction has not completed within the specified timeout (dbms.transaction.timeout). You may want to retry with a longer timeout.
  • 13:06:41.000 (infos) Importing 128000 rows from new_traits_1645545266_1645721181_chunk_9.csv
  • 12:21:39.000 (infos) Waiting 2700 seconds for the last 'chunk' to be added to neo4j...
  • 12:20:10.000 (infos) Importing 128000 rows from new_traits_1645545266_1645721181_chunk_8.csv
  • 12:00:09.000 (infos) Waiting 1200 seconds for the last 'chunk' to be added to neo4j...
  • 11:58:53.000 (infos) Importing 128000 rows from new_traits_1645545266_1645721181_chunk_7.csv
  • 11:41:22.000 (infos) Waiting 1050 seconds for the last 'chunk' to be added to neo4j...
  • 11:40:14.000 (infos) Importing 128000 rows from new_traits_1645545266_1645721181_chunk_6.csv
  • 11:25:13.000 (infos) Waiting 900 seconds for the last 'chunk' to be added to neo4j...
  • 11:23:51.000 (infos) Importing 128000 rows from new_traits_1645545266_1645721181_chunk_5.csv
  • 11:11:15.000 (infos) Waiting 750 seconds for the last 'chunk' to be added to neo4j...
  • 11:09:56.000 (infos) Importing 128000 rows from new_traits_1645545266_1645721181_chunk_4.csv
  • 10:59:55.000 (infos) Waiting 600 seconds for the last 'chunk' to be added to neo4j...
  • 10:57:31.000 (infos) Importing 128000 rows from new_traits_1645545266_1645721181_chunk_3.csv
  • 10:50:01.000 (infos) Waiting 450 seconds for the last 'chunk' to be added to neo4j...
  • 10:48:40.000 (infos) Importing 128000 rows from new_traits_1645545266_1645721181_chunk_2.csv
  • 10:43:39.000 (infos) Waiting 300 seconds for the last 'chunk' to be added to neo4j...
  • 10:42:28.000 (infos) Importing 128000 rows from new_traits_1645545266_1645721181_chunk_1.csv
  • 10:42:27.000 (warns) Found 2140044 rows, will break up into 128000
  • 10:42:26.000 (infos) adding new traits