-
Started:
April 28, 2024 12:13
-
Completed:
17:53:22.000
-
Failed:
No.
-
Status:
completed
Events (most recent first):
-
09:55:18.000
(infos)
resources_controller.rb:100>resource.rb:505>content_server_connection.rb:106>pub_log.rb:48= Connecting to https://content.eol.org/ ...
-
09:54:53.000
(infos)
resources_controller.rb:100>resource.rb:505>content_server_connection.rb:106>pub_log.rb:48= Downloaded: 1 files, 932M in 27s (34.8 MB/s)
-
09:54:53.000
(infos)
resources_controller.rb:100>resource.rb:505>content_server_connection.rb:106>pub_log.rb:48= FINISHED --2024-05-26 13:54:53--
-
09:54:53.000
(infos)
resources_controller.rb:100>resource.rb:505>content_server_connection.rb:106>pub_log.rb:48= 2024-05-26 13:54:53 (34.8 MB/s) - ‘/app/tmp/try_summarized_r_tmp_1716731666_publish_traits.tsv’ saved [976844777/976844777]
-
09:54:53.000
(infos)
resources_controller.rb:100>resource.rb:505>content_server_connection.rb:106>pub_log.rb:48= Saving to: ‘/app/tmp/try_summarized_r_tmp_1716731666_publish_traits.tsv’
-
09:54:53.000
(infos)
resources_controller.rb:100>resource.rb:505>content_server_connection.rb:106>pub_log.rb:48= Length: 976844777 (932M) [application/octet-stream]
-
09:54:53.000
(infos)
resources_controller.rb:100>resource.rb:505>content_server_connection.rb:106>pub_log.rb:48= HTTP request sent, awaiting response... 200 OK
-
09:54:53.000
(infos)
resources_controller.rb:100>resource.rb:505>content_server_connection.rb:106>pub_log.rb:48= Connecting to content.eol.org (content.eol.org)|160.111.248.42|:443... connected.
-
09:54:53.000
(infos)
resources_controller.rb:100>resource.rb:505>content_server_connection.rb:106>pub_log.rb:48= Resolving content.eol.org (content.eol.org)... 160.111.248.42
-
09:54:53.000
(warns)
resources_controller.rb:100>resource.rb:505>content_server_connection.rb:114>pub_log.rb:48= Took 27 seconds.
-
09:54:26.000
(warns)
resources_controller.rb:100>resource.rb:505>content_server_connection.rb:114>pub_log.rb:48= USING wget TO RETRIEVE FULL FILE...
-
09:54:26.000
(warns)
resources_controller.rb:100>resource.rb:505>content_server_connection.rb:114>pub_log.rb:48= TRUNCATED RESPONSE! Got 345927151 bytes out of 976844777 from https://content.eol.org/data/try_summarized_r/publish_traits.tsv
-
09:54:17.000
(infos)
resources_controller.rb:100>resource.rb:505>content_server_connection.rb:106>pub_log.rb:48= Connecting to https://content.eol.org/ ...
-
09:54:14.000
(infos)
resources_controller.rb:100>resource.rb:505>content_server_connection.rb:106>pub_log.rb:48= polling for trait diff metadata: /resources/578/publish_diffs.json?since=1714514002
-
17:53:22.000
(ends)
resource.rb:519:in `republish_traits'> fast.rb:195:in `traits_by_resource'> Complete
-
17:53:22.000
(ends)
resource.rb:519:in `republish_traits'> fast.rb:194:in `traits_by_resource'> TOTAL TIME: 82.1m
-
17:53:22.000
(infos)
fast.rb:334:in `publish_traits'> slurp.rb:36:in `load_resource_from_repo'> Removing trait and metadata files
-
17:53:22.000
(infos)
fast.rb:334:in `publish_traits'> slurp.rb:351:in `log_node_count'> Nodes: 514734; Traits: 509455; MetaData: 455393
-
17:52:54.000
(warns)
fast.rb:334:in `publish_traits'> slurp.rb:289:in `rescue in block (2 levels) in load_csv'> ...re-trying.
-
17:47:54.000
(warns)
fast.rb:334:in `publish_traits'> slurp.rb:285:in `rescue in block (2 levels) in load_csv'> FAILED on build_nodes query (MetaData), will re-try 3 times after 5 minute pause (the site may be too busy to serve the CSV to Neo4j)...
-
17:47:54.000
(warns)
fast.rb:334:in `publish_traits'> slurp.rb:284:in `rescue in block (2 levels) in load_csv'> At https://eol.org/data/try_summarized_r/publish_metadata_chunk_4.csv @ position 21505559 - Missing end for quote (") which started on line 73407
-
17:47:54.000
(warns)
fast.rb:334:in `publish_traits'> slurp.rb:571:in `rescue in autocommit_query'> Exception (Neo4j::Driver::Exceptions::DatabaseException) QUERY: {USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'https://eol.org/data/try_summarized_r/publish_metadata_chunk_4.csv' AS row WITH row WHERE 1=1 MERGE (metadata:MetaData { eol_pk: row.eol_pk })
ON CREATE SET metadata.source = row.source
ON MATCH SET metadata.source = row.source
ON CREATE SET metadata.literal = row.literal
ON MATCH SET metadata.literal = row.literal
ON CREATE SET metadata.measurement = row.measurement
ON MATCH SET metadata.measurement = row.measurement} MESSAGE: At https://eol.org/data/try_summarized_r/publish_metadata_chunk_4.csv @ position 21505559 - Missing end for quote (") which started on line 73407
-
17:47:49.000
(infos)
fast.rb:334:in `publish_traits'> slurp.rb:275:in `block in load_csv'> Importing 128000 rows from publish_metadata_chunk_4.csv
-
17:47:13.000
(warns)
each.rb:9:in `each'> slurp.rb:289:in `rescue in block (2 levels) in load_csv'> ...re-trying.
-
17:42:13.000
(warns)
each.rb:9:in `each'> slurp.rb:285:in `rescue in block (2 levels) in load_csv'> FAILED on build_nodes query (MetaData), will re-try 2 times after 5 minute pause (the site may be too busy to serve the CSV to Neo4j)...
-
17:42:13.000
(warns)
each.rb:9:in `each'> slurp.rb:284:in `rescue in block (2 levels) in load_csv'> At https://eol.org/data/try_summarized_r/publish_metadata_chunk_3.csv @ position 14382018 - Missing end for quote (") which started on line 49713
-
17:42:13.000
(warns)
each.rb:9:in `each'> slurp.rb:571:in `rescue in autocommit_query'> Exception (Neo4j::Driver::Exceptions::DatabaseException) QUERY: {USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'https://eol.org/data/try_summarized_r/publish_metadata_chunk_3.csv' AS row WITH row WHERE 1=1 MERGE (metadata:MetaData { eol_pk: row.eol_pk })
ON CREATE SET metadata.source = row.source
ON MATCH SET metadata.source = row.source
ON CREATE SET metadata.literal = row.literal
ON MATCH SET metadata.literal = row.literal
ON CREATE SET metadata.measurement = row.measurement
ON MATCH SET metadata.measurement = row.measurement} MESSAGE: At https://eol.org/data/try_summarized_r/publish_metadata_chunk_3.csv @ position 14382018 - Missing end for quote (") which started on line 49713
-
17:42:12.000
(warns)
each.rb:9:in `each'> slurp.rb:289:in `rescue in block (2 levels) in load_csv'> ...re-trying.
-
17:37:12.000
(warns)
each.rb:9:in `each'> slurp.rb:285:in `rescue in block (2 levels) in load_csv'> FAILED on build_nodes query (MetaData), will re-try 3 times after 5 minute pause (the site may be too busy to serve the CSV to Neo4j)...
-
17:37:12.000
(warns)
each.rb:9:in `each'> slurp.rb:284:in `rescue in block (2 levels) in load_csv'> At https://eol.org/data/try_summarized_r/publish_metadata_chunk_3.csv @ position 10629745 - Missing end for quote (") which started on line 36531
-
17:37:12.000
(warns)
each.rb:9:in `each'> slurp.rb:571:in `rescue in autocommit_query'> Exception (Neo4j::Driver::Exceptions::DatabaseException) QUERY: {USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'https://eol.org/data/try_summarized_r/publish_metadata_chunk_3.csv' AS row WITH row WHERE 1=1 MERGE (metadata:MetaData { eol_pk: row.eol_pk })
ON CREATE SET metadata.source = row.source
ON MATCH SET metadata.source = row.source
ON CREATE SET metadata.literal = row.literal
ON MATCH SET metadata.literal = row.literal
ON CREATE SET metadata.measurement = row.measurement
ON MATCH SET metadata.measurement = row.measurement} MESSAGE: At https://eol.org/data/try_summarized_r/publish_metadata_chunk_3.csv @ position 10629745 - Missing end for quote (") which started on line 36531
-
17:37:10.000
(infos)
each.rb:9:in `each'> slurp.rb:275:in `block in load_csv'> Importing 128000 rows from publish_metadata_chunk_3.csv
-
17:31:09.000
(infos)
each.rb:9:in `each'> slurp.rb:329:in `block in break_up_large_file'> Waiting 6 minutes for the part 3 of 4 to be added to neo4j.
-
17:31:09.000
(infos)
each.rb:9:in `each'> slurp.rb:351:in `log_node_count'> Nodes: 514734; Traits: 509455; MetaData: 256000
-
17:30:31.000
(infos)
each.rb:9:in `each'> slurp.rb:275:in `block in load_csv'> Importing 128000 rows from publish_metadata_chunk_2.csv
-
17:26:30.000
(infos)
each.rb:9:in `each'> slurp.rb:329:in `block in break_up_large_file'> Waiting 4 minutes for the part 2 of 4 to be added to neo4j.
-
17:26:30.000
(infos)
each.rb:9:in `each'> slurp.rb:351:in `log_node_count'> Nodes: 514734; Traits: 509455; MetaData: 128000
-
17:25:54.000
(warns)
each.rb:9:in `each'> slurp.rb:289:in `rescue in block (2 levels) in load_csv'> ...re-trying.
-
17:20:54.000
(warns)
each.rb:9:in `each'> slurp.rb:285:in `rescue in block (2 levels) in load_csv'> FAILED on build_nodes query (MetaData), will re-try 2 times after 5 minute pause (the site may be too busy to serve the CSV to Neo4j)...
-
17:20:54.000
(warns)
each.rb:9:in `each'> slurp.rb:284:in `rescue in block (2 levels) in load_csv'> At https://eol.org/data/try_summarized_r/publish_metadata_chunk_1.csv @ position 19695735 - Missing end for quote (") which started on line 62521
-
17:20:54.000
(warns)
each.rb:9:in `each'> slurp.rb:571:in `rescue in autocommit_query'> Exception (Neo4j::Driver::Exceptions::DatabaseException) QUERY: {USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'https://eol.org/data/try_summarized_r/publish_metadata_chunk_1.csv' AS row WITH row WHERE 1=1 MERGE (metadata:MetaData { eol_pk: row.eol_pk })
ON CREATE SET metadata.source = row.source
ON MATCH SET metadata.source = row.source
ON CREATE SET metadata.literal = row.literal
ON MATCH SET metadata.literal = row.literal
ON CREATE SET metadata.measurement = row.measurement
ON MATCH SET metadata.measurement = row.measurement} MESSAGE: At https://eol.org/data/try_summarized_r/publish_metadata_chunk_1.csv @ position 19695735 - Missing end for quote (") which started on line 62521
-
17:20:50.000
(warns)
each.rb:9:in `each'> slurp.rb:289:in `rescue in block (2 levels) in load_csv'> ...re-trying.
-
17:15:50.000
(warns)
each.rb:9:in `each'> slurp.rb:285:in `rescue in block (2 levels) in load_csv'> FAILED on build_nodes query (MetaData), will re-try 3 times after 5 minute pause (the site may be too busy to serve the CSV to Neo4j)...
-
17:15:50.000
(warns)
each.rb:9:in `each'> slurp.rb:284:in `rescue in block (2 levels) in load_csv'> At https://eol.org/data/try_summarized_r/publish_metadata_chunk_1.csv @ position 23792933 - Missing end for quote (") which started on line 75250
-
17:15:50.000
(warns)
each.rb:9:in `each'> slurp.rb:571:in `rescue in autocommit_query'> Exception (Neo4j::Driver::Exceptions::DatabaseException) QUERY: {USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'https://eol.org/data/try_summarized_r/publish_metadata_chunk_1.csv' AS row WITH row WHERE 1=1 MERGE (metadata:MetaData { eol_pk: row.eol_pk })
ON CREATE SET metadata.source = row.source
ON MATCH SET metadata.source = row.source
ON CREATE SET metadata.literal = row.literal
ON MATCH SET metadata.literal = row.literal
ON CREATE SET metadata.measurement = row.measurement
ON MATCH SET metadata.measurement = row.measurement} MESSAGE: At https://eol.org/data/try_summarized_r/publish_metadata_chunk_1.csv @ position 23792933 - Missing end for quote (") which started on line 75250
-
17:15:47.000
(infos)
each.rb:9:in `each'> slurp.rb:275:in `block in load_csv'> Importing 128000 rows from publish_metadata_chunk_1.csv
-
17:15:46.000
(warns)
fast.rb:334:in `publish_traits'> slurp.rb:308:in `break_up_large_files'> Found 477241 rows, will break up into 4 of 128000
-
17:15:46.000
(infos)
fast.rb:334:in `publish_traits'> slurp.rb:592:in `add_metadata'> adding new metadata
-
17:15:46.000
(infos)
fast.rb:334:in `publish_traits'> slurp.rb:351:in `log_node_count'> Nodes: 514734; Traits: 509455; MetaData: 0
-
17:15:41.000
(infos)
fast.rb:334:in `publish_traits'> slurp.rb:275:in `block in load_csv'> Importing 128000 rows from publish_traits_chunk_5.csv
