1200字范文,内容丰富有趣,写作的好帮手!
1200字范文 > python数据格式化后导入数据库_如何为数据库导入重新格式化这个json?

python数据格式化后导入数据库_如何为数据库导入重新格式化这个json?

时间:2019-04-25 10:08:08

相关推荐

python数据格式化后导入数据库_如何为数据库导入重新格式化这个json?

我有这些“json”文件,我想插入到我的mongodb数据库中。你知道吗

问题是,它的格式如下:{ "channelType":"TEMPGROUP", ... } # line 1

{ "channelType":"TEMPGROUP", ... } # line 2

因此,它不是将它作为一个文档插入数据库,而是将每一行作为一个条目插入。最终,数据库中3个“json”文件中的3个文档变成了数据库中的1189个文档。你知道吗

如何将“.json”的全部内容插入到一个文档中?你知道吗

我的代码是:replay_url = "/live/getreplayvideos?"

userid = 969730808384462848

url2 = replay_url + urllib.parse.urlencode({'userid': userid}) + '&page_size=1000'

raw_replay_data = requests.get(url2).json()

for i in raw_replay_data['data']['video_info']:

url3 = i['msgfile']

raw_message_data = urllib.request.urlopen(url3)

for line in raw_message_data:

json_data = json.loads(line)

messages.insert_one(json_data)

print(json_data)

更新以提供更多回答信息

在消息.插入(json\ U数据)给出以下错误:Traceback (most recent call last):

File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/collection.py", line 633, in _insert

blk.execute(concern, session=session)

File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/bulk.py", line 432, in execute

return self.execute_command(generator, write_concern, session)

File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/bulk.py", line 329, in execute_command

raise BulkWriteError(full_result)

pymongo.errors.BulkWriteError: batch op errors occurred

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/import_messages_dev.py", line 43, in

messages.insert(json_data)

File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/collection.py", line 2941, in insert

check_keys, manipulate, write_concern)

File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/collection.py", line 635, in _insert

_raise_last_error(bwe.details)

File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/helpers.py", line 220, in _raise_last_error

_raise_last_write_error(write_errors)

File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/helpers.py", line 188, in _raise_last_write_error

raise DuplicateKeyError(error.get("errmsg"), 11000, error)

pymongo.errors.DuplicateKeyError: E11000 duplicate key error index: liveme.messages.$_id_ dup key: { : ObjectId('5aa2fc6f5d60126499060949') }

在消息。插入一个(json\ U数据)给了我这个错误:Traceback (most recent call last):

File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/import_messages_dev.py", line 43, in

messages.insert_one(json_data)

File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/collection.py", line 676, in insert_one

common.validate_is_document_type("document", document)

File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/common.py", line 434, in validate_is_document_type

"collections.MutableMapping" % (option,))

TypeError: document must be an instance of dict, bson.son.SON, bson.raw_bson.RawBSONDocument, or a type that inherits from collections.MutableMapping

在消息。插入\u多个(json\ U数据)给了我这个错误:Traceback (most recent call last):

File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/import_messages_dev.py", line 43, in

messages.insert_many(json_data)

File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/collection.py", line 742, in insert_many

blk.execute(self.write_concern.document, session=session)

File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/bulk.py", line 432, in execute

return self.execute_command(generator, write_concern, session)

File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/bulk.py", line 329, in execute_command

raise BulkWriteError(full_result)

pymongo.errors.BulkWriteError: batch op errors occurred

在消息.插入以及消息。插入\u多个两者都插入一行并抛出错误。你知道吗

本内容不代表本网观点和政治立场,如有侵犯你的权益请联系我们处理。
网友评论
网友评论仅供其表达个人看法,并不表明网站立场。