pymongo


mongoengine slow serialization of embedded documents with reference fields


I have a small DB with about 500 records. I'm trying to implement a versioning scheme where I save the form along with its current version to my Record collection. Ideally, I would like to store the form along with its version number in an embedded document to keep things nice and tidy:
class Structure(db.EmbeddedDocument):
form = db.ReferenceField(Form, required = True)
version = db.IntField(required = True)
#property
def short(self):
return {
'form': self.form,
'version': self.version
}
class Record(db.Document):
structure = db.EmbeddedDocumentField(Structure)
#property
def short(self):
return {
'structure': self.structure.short
}
This way when I recall a record I can grab the form and the version that was used at the time. Running some timing tests:
start = time.clock()
records = Record.objects.select_related()
print ('Time: ', time.clock() - start)
response = [i.short for i in records]
print ('Time: ', time.clock() - start)
I find the query time for all records Record.objects.select_related() to be reasonable at, ~ 1.12 s, however, I'm finding serialization for the purpose of JSON transfer is extremely expensive at ~ 24.1s!
If I make a slight modification by removing use of the EmbeddedDocument:
class Record(db.Document):
form = db.ReferenceField(Form, required = True)
version = db.IntField(required = True)
#property
def short(self):
return {
'form': self.form,
'version': self.version
}
Running the same test I find the query time to be pretty much unchanged at ~ 1.36 s, however, the serialization time improved by 24x to 1.14s. I really do not understand why use of an embedded document would lead to such as massive penalty in serialization time...? Is dereferencing in an embedded object more difficult?

Related Links

pymongo query return in direct query not in pymongo
Displaying nested dictionaries in flask-admin
mongoengine slow serialization of embedded documents with reference fields
Pymongo error : 'Cursor' object has no attribute 'find' , how to filter it?
Spark Streaming: foreachRDD insert into mongoDB using python?
how to find right version of bson from pip for pymongo/mongoengine
How to enable `projection` in pymongo query class `find()`?
Variables don't hold value for very long pymongo ipython
pymongo upgrade to 3.0.3 causes this statement to fail
UUID to NUUID in Python
how to test for empty mongodb query in pymong
read video file from mongodb with pymongo
Get OperationFailure error with PyMongo 2.7.1
Inserting string as regular string in mongodb
Mongos + Pymongo 2.5 ==>No suitable hosts found
pymongo collection add item using update

Categories

HOME
jboss
memory
svn
typelite
vagrant
openxml
android-fragments
redmine
selenium-builder
adobe
visual-studio-2010
magento-2.0
informatica
nsis
elisp
material-components
repair
sentry
position
angular-meteor
jpa-criteria
log4j2
yahoo
bootstrap-switch
exe
tfsbuild
wsf
sybase-ase
ava
multi-dimensional-scaling
list-comprehension
ccavenue
guice
monitor
phpstorm-2016.1
python-2.x
shapeless
markov-chains
android-canvas
des
numerics
djcelery
right-click
get-event-store
icepdf
distributed-caching
flume-twitter
axis
roracle
watir-webdriver
emf-compare
d3v4
multi-touch
clp
visual-studio-monaco
oci
android-4.2-jelly-bean
android-sdcard
noclassdeffounderror
strpos
simian
i2b2
enunciate
outlook.com
phpquery
in-app-billing
variable-length-array
directx-9
mixins
360-degrees
supersocket.net
tree-structure
python-c-extension
n-tier-architecture
password-recovery
ildasm
lumia-imaging-sdk
prism.js
oxygene
spring-social-linkedin
smips
vlab
arel
oscilloscope
cmath
itunes-sdk
semantic-merge
multiscaleimage
instance-variables
glx
delphi-prism
haiku
upshot
airprint
jmenu
active-record-query
pinax
sscli
google-translator-toolkit
ppc
fxruby
xap
twitter-feed
community-server
webclient.uploaddata

Resources

Database Users
RDBMS discuss
Database Dev&Adm
javascript
java
csharp
php
android
javascript
java
csharp
php
python
android
jquery
ruby
ios
html
Mobile App
Mobile App
Mobile App