pymongo


mongoengine slow serialization of embedded documents with reference fields


I have a small DB with about 500 records. I'm trying to implement a versioning scheme where I save the form along with its current version to my Record collection. Ideally, I would like to store the form along with its version number in an embedded document to keep things nice and tidy:
class Structure(db.EmbeddedDocument):
form = db.ReferenceField(Form, required = True)
version = db.IntField(required = True)
#property
def short(self):
return {
'form': self.form,
'version': self.version
}
class Record(db.Document):
structure = db.EmbeddedDocumentField(Structure)
#property
def short(self):
return {
'structure': self.structure.short
}
This way when I recall a record I can grab the form and the version that was used at the time. Running some timing tests:
start = time.clock()
records = Record.objects.select_related()
print ('Time: ', time.clock() - start)
response = [i.short for i in records]
print ('Time: ', time.clock() - start)
I find the query time for all records Record.objects.select_related() to be reasonable at, ~ 1.12 s, however, I'm finding serialization for the purpose of JSON transfer is extremely expensive at ~ 24.1s!
If I make a slight modification by removing use of the EmbeddedDocument:
class Record(db.Document):
form = db.ReferenceField(Form, required = True)
version = db.IntField(required = True)
#property
def short(self):
return {
'form': self.form,
'version': self.version
}
Running the same test I find the query time to be pretty much unchanged at ~ 1.36 s, however, the serialization time improved by 24x to 1.14s. I really do not understand why use of an embedded document would lead to such as massive penalty in serialization time...? Is dereferencing in an embedded object more difficult?

Related Links

UUID to NUUID in Python
how to test for empty mongodb query in pymong
read video file from mongodb with pymongo
Get OperationFailure error with PyMongo 2.7.1
Inserting string as regular string in mongodb
Mongos + Pymongo 2.5 ==>No suitable hosts found
pymongo collection add item using update
How to convert objectid to string
Conditional atomic mongo remove with different results
python script to connect to mongodb master
flask-admin and pymongo search
Running code when mongoengine loads a document
Can't replace mongo document
How do I call a function with pymongo using webpy?
python motor offers fetch_next exception on aggregation
python motor mongo cursor length or peek next

Categories

HOME
orientdb
long-integer
activex
systemd
code-formatting
odata
tibco
jint
video-streaming
alljoyn
erd
fbloginview
game-maker
trace32
django-rq
playframework-2.0
string-formatting
spring-shell
coreldraw
maven-surefire-plugin
fluent-migrator
sybase-ase
macromedia
php-5.3
bitbake
netbios
snap-framework
mongodb-3.4
universal
sql-execution-plan
ms-media-foundation
xmlunit
transform
textmate2
utf
system-on-chip
angular-datatables
permission-denied
absolute
realstudio
ol3-google-maps
lightning-workbench
node-apn
scala-collections
patternlab.io
degrees
gawk
allegro
linuxbrew
android-mapview
arbre
chicagoboss
canopy
ngcordova
bootstrap-wysiwyg
artisan
wicked-gem
debug-symbols
incognito-mode
air-native-extension
gpu-programming
django-sites
assetic
particle-swarm
facebook-game-groups
ntlmv2
linkbutton
arrow
drawimage
photo-management
ognl
hungarian-notation
uninstaller
yahoo-maps
html-components
xap
accumulator
database-cloning
index.dat
error-detection

Resources

Mobile Apps Dev
Database Users
javascript
java
csharp
php
android
MS Developer
developer works
python
ios
c
html
jquery
RDBMS discuss
Cloud Virtualization
Database Dev&Adm
javascript
java
csharp
php
python
android
jquery
ruby
ios
html
Mobile App
Mobile App
Mobile App