Styjun

Assume that there is simple User and Post model.

class User(Document):
 user_id = StringField(primary_key=True)
 gender = StringField(default='M')


class Post(Document):
 user = ReferenceField(User)
 body = StringField()


if __name__ == '__main__':
 hide = User(user_id='hide', gender='M').save()
 john = User(user_id='john', gender='M').save()
 test = User(user_id='test', gender='W').save()
 admin = User(user_id='admin', gender='W').save()

 Post(user=hide, body='hide post').save()
 Post(user=john, body='john post').save()
 Post(user=test, body='test post').save()
 Post(user=admin, body='admin post').save()

 hide = User.objects(user_id='hide').first()

 posts = Post.objects(user__ne=hide)
 for post in posts:
 print(post.body)

Result is

john post
test post
admin post

I triggered condition user__ne=hide, So every posts printed except hide's post.

In this case, How can I add more condition likes, gender='W'?

Below code is result of I tried.

posts = Post.objects(user__ne=hide, user__gender__ne='M')

and

from mongoengine.queryset.visitor import Q

posts = Post.objects(Q(user__ne=hide) & Q(user__gender__ne='M'))

But both code throw errors -> mongoengine.errors.InvalidQueryError: Cannot perform join in mongoDB: user__gender

I know it can be implemented with this.

gender = User.objects(gender__ne='M')
posts = Post.objects(Q(user__ne=hide) & Q(user__nin=gender))

But if users has too many rows, it maybe occur memory issue.

Question

1. Is it possible to querying with condition at once?




2. Do .objects() actually querying to database?

As detailed on the sibling github ticket

1 - There is no joins in mongodb so there is no other option than the one you suggested. One simple thing you can do to improve performance and reduce the memory footprint is to fetch only the user ids, see below:

male_ids = User.objects(gender__ne='M').scalar('id') # Only fetch the user ids, i.o loading full object data into User model
posts = Post.objects(Q(user__ne=hide) & Q(user__nin=male_ids))

Note: There is a CachedReferenceField in mongoengine that might help you to achieve what you want (it would basically duplicates the value of gender next to the reference to the user in the Post collection) and keep it in sync but CachedReferenceField suffers from some bugs (and performance issue to keep them in sync) so maybe it could do the trick for simple use cases but I wouldn't advise to use it.

2 - .objects() returns a queryset, the query is only fired when you iterate over the queryset (or print the queryset). See below:

user_qs = User.objects()
print(type(user_qs)) # <class mongoengine.queryset.queryset.QuerySet>, query not fired yet
for user in qs_user: # fires the actual query and load data in User instances
 pass