iNaturalist data coverage的匯入日誌

Started: May 24, 2024 02:00
Completed: 03:35:32
Failed: No.
Status: completed

Events (most recent first):

顯示警告（21 個）

1
2
3
4
Last »

03:35:32 (ends) resource.rb:225>fast.rb:177>pub_log.rb:73>import_log.rb:89= Complete
03:35:32 (ends) delayed_job.rb:17>resource.rb:225>fast.rb:176>pub_log.rb:48= TOTAL TIME: 1.6h
03:35:32 (starts) delayed_job.rb:17>resource.rb:225>fast.rb:163>pub_log.rb:48= TraitBank::Denormalizer.update_resource_vernaculars
03:35:32 (starts) resource.rb:225>fast.rb:163>pub_log.rb:20>import_log.rb:82= Running
03:35:28 (starts) delayed_job.rb:17>resource.rb:225>fast.rb:161>pub_log.rb:48= Resource#fix_native_nodes
03:35:28 (starts) resource.rb:225>fast.rb:161>pub_log.rb:20>import_log.rb:82= Running
03:35:28 (infos) delayed_job.rb:17>resource.rb:225>fast.rb:235>pub_log.rb:48= Removing /app/tmp/i_nat_dat_covera_node_ancestors.tsv
03:35:28 (infos) delayed_job.rb:17>resource.rb:225>fast.rb:235>pub_log.rb:48= Removing /app/tmp/i_nat_dat_covera_scientific_names.tsv
03:35:28 (infos) delayed_job.rb:17>resource.rb:225>fast.rb:235>pub_log.rb:48= Removing /app/tmp/i_nat_dat_covera_nodes.tsv
03:35:28 (infos) resource.rb:225>fast.rb:334>slurp.rb:36>pub_log.rb:48= Removing trait and metadata files
03:35:28 (infos) resource.rb:225>fast.rb:334>slurp.rb:351>pub_log.rb:48= Nodes: 625201; Traits: 522047; MetaData: 0
03:35:25 (infos) resource.rb:225>fast.rb:334>slurp.rb:275>pub_log.rb:48= Importing 128000 rows from publish_metadata.tsv
03:35:25 (infos) resource.rb:225>fast.rb:334>slurp.rb:592>pub_log.rb:48= adding new metadata
03:35:25 (infos) resource.rb:225>fast.rb:334>slurp.rb:351>pub_log.rb:48= Nodes: 625201; Traits: 522047; MetaData: 0
03:34:59 (infos) resource.rb:225>fast.rb:334>slurp.rb:275>pub_log.rb:48= Importing 128000 rows from publish_traits_chunk_7.csv
03:34:10 (infos) fast.rb:334>each.rb:9>slurp.rb:275>pub_log.rb:48= Importing 128000 rows from publish_traits_chunk_6.csv
03:22:09 (infos) fast.rb:334>each.rb:9>slurp.rb:329>pub_log.rb:48= Waiting 12 minutes for the part 6 of 7 to be added to neo4j.
03:22:09 (infos) fast.rb:334>each.rb:9>slurp.rb:351>pub_log.rb:48= Nodes: 496904; Traits: 448645; MetaData: 0
03:20:56 (infos) fast.rb:334>each.rb:9>slurp.rb:275>pub_log.rb:48= Importing 128000 rows from publish_traits_chunk_5.csv
03:10:56 (infos) fast.rb:334>each.rb:9>slurp.rb:329>pub_log.rb:48= Waiting 10 minutes for the part 5 of 7 to be added to neo4j.
03:10:56 (infos) fast.rb:334>each.rb:9>slurp.rb:351>pub_log.rb:48= Nodes: 368904; Traits: 323785; MetaData: 0
03:09:42 (infos) fast.rb:334>each.rb:9>slurp.rb:275>pub_log.rb:48= Importing 128000 rows from publish_traits_chunk_4.csv
03:01:41 (infos) fast.rb:334>each.rb:9>slurp.rb:329>pub_log.rb:48= Waiting 8 minutes for the part 4 of 7 to be added to neo4j.
03:01:41 (infos) fast.rb:334>each.rb:9>slurp.rb:351>pub_log.rb:48= Nodes: 240904; Traits: 197650; MetaData: 0
03:00:53 (infos) fast.rb:334>each.rb:9>slurp.rb:275>pub_log.rb:48= Importing 128000 rows from publish_traits_chunk_3.csv
02:54:52 (infos) fast.rb:334>each.rb:9>slurp.rb:329>pub_log.rb:48= Waiting 6 minutes for the part 3 of 7 to be added to neo4j.
02:54:52 (infos) fast.rb:334>each.rb:9>slurp.rb:351>pub_log.rb:48= Nodes: 151774; Traits: 108520; MetaData: 0
02:54:01 (warns) fast.rb:334>each.rb:9>slurp.rb:289>pub_log.rb:48= ...re-trying.
02:49:01 (warns) fast.rb:334>each.rb:9>slurp.rb:285>pub_log.rb:48= FAILED on build_nodes query (Page), will re-try 2 times after 5 minute pause (the site may be too busy to serve the CSV to Neo4j)...
02:49:01 (warns) fast.rb:334>each.rb:9>slurp.rb:284>pub_log.rb:48= Cannot merge the following node because of null property value for 'page_id': (page:Page {page_id: null}) (Failure when processing file '/data/i_nat_dat_covera/publish_traits_chunk_2.csv' on line 57828 (which is the last row in the file).)
02:49:01 (warns) fast.rb:334>each.rb:9>slurp.rb:571>pub_log.rb:48= Exception (Neo4j::Driver::Exceptions::ClientException) QUERY: {USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'https://eol.org/data/i_nat_dat_covera/publish_traits_chunk_2.csv' AS row WITH row WHERE 1=1 MERGE (page:Page { page_id: toInteger(row.page_id) })} MESSAGE: Cannot merge the following node because of null property value for 'page_id': (page:Page {page_id: null}) (Failure when processing file '/data/i_nat_dat_covera/publish_traits_chunk_2.csv' on line 57828 (which is the last row in the file).)
02:48:57 (warns) fast.rb:334>each.rb:9>slurp.rb:289>pub_log.rb:48= ...re-trying.
02:43:57 (warns) fast.rb:334>each.rb:9>slurp.rb:285>pub_log.rb:48= FAILED on build_nodes query (Page), will re-try 3 times after 5 minute pause (the site may be too busy to serve the CSV to Neo4j)...
02:43:57 (warns) fast.rb:334>each.rb:9>slurp.rb:284>pub_log.rb:48= Cannot merge the following node because of null property value for 'page_id': (page:Page {page_id: null}) (Failure when processing file '/data/i_nat_dat_covera/publish_traits_chunk_2.csv' on line 30221 (which is the last row in the file).)
02:43:57 (warns) fast.rb:334>each.rb:9>slurp.rb:571>pub_log.rb:48= Exception (Neo4j::Driver::Exceptions::ClientException) QUERY: {USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'https://eol.org/data/i_nat_dat_covera/publish_traits_chunk_2.csv' AS row WITH row WHERE 1=1 MERGE (page:Page { page_id: toInteger(row.page_id) })} MESSAGE: Cannot merge the following node because of null property value for 'page_id': (page:Page {page_id: null}) (Failure when processing file '/data/i_nat_dat_covera/publish_traits_chunk_2.csv' on line 30221 (which is the last row in the file).)
02:43:54 (infos) fast.rb:334>each.rb:9>slurp.rb:275>pub_log.rb:48= Importing 128000 rows from publish_traits_chunk_2.csv
02:39:53 (infos) fast.rb:334>each.rb:9>slurp.rb:329>pub_log.rb:48= Waiting 4 minutes for the part 2 of 7 to be added to neo4j.
02:39:53 (infos) fast.rb:334>each.rb:9>slurp.rb:351>pub_log.rb:48= Nodes: 106246; Traits: 62992; MetaData: 0
02:38:56 (infos) fast.rb:334>each.rb:9>slurp.rb:275>pub_log.rb:48= Importing 128000 rows from publish_traits_chunk_1.csv
02:38:55 (warns) resource.rb:225>fast.rb:334>slurp.rb:308>pub_log.rb:48= Found 811042 rows, will break up into 7 of 128000
02:38:55 (infos) resource.rb:225>fast.rb:334>slurp.rb:582>pub_log.rb:48= adding new traits
02:38:55 (infos) resource.rb:225>fast.rb:334>slurp.rb:611>pub_log.rb:48= not removing any traits
02:38:55 (infos) fast.rb:334>slurp.rb:20>content_server_connection.rb:106>pub_log.rb:48= Connecting to https://content.eol.org/ ...
02:38:49 (infos) fast.rb:334>slurp.rb:20>content_server_connection.rb:106>pub_log.rb:48= Connecting to https://content.eol.org/ ...
02:38:49 (infos) fast.rb:334>slurp.rb:20>content_server_connection.rb:106>pub_log.rb:48= polling for trait diff metadata: /resources/1219/publish_diffs.json
02:38:49 (starts) delayed_job.rb:17>resource.rb:225>fast.rb:202>pub_log.rb:48= #publish_traits = TraitBank::Slurp.load_resource_from_repo
02:38:49 (starts) resource.rb:225>fast.rb:202>pub_log.rb:20>import_log.rb:82= Running
02:38:49 (warns) resource.rb:225>fast.rb:155>page_creator.rb:22>pub_log.rb:48= There were NO new pages, skipping...
02:37:23 (starts) resource.rb:225>fast.rb:155>page_creator.rb:5>pub_log.rb:48= create_new_pages
02:37:22 (starts) delayed_job.rb:17>resource.rb:225>fast.rb:154>pub_log.rb:48= PageCreator

1
2
3
4
Last »