Aggregate $lookup with C#MongoDb - Joining ObjectId references in the list with related collectionsUnable to join two collection where condition exist in subcollectionMongo linq query left outer joinAggregate $lookup Total size of documents in matching pipeline exceeds maximum document sizeHow to $lookup with MongoDB C# driver?Modeling mongodb stored referenced relationships in .net coreSystem.ArgumentException with C# MongoDB.DriverHow do I calculate someone's age in C#?What is the difference between String and string in C#?Hidden Features of C#?Cast int to enum in C#How do you give a C# Auto-Property a default value?How do I enumerate an enum in C#?What are the correct version numbers for C#?How do I get a consistent byte representation of strings in C# without manually specifying an encoding?Get int value from enum in C#How to join two collections using lookup operator in C# Mongodb strongly typed driver
Generating function of ordered partitions
Identification of vintage sloping window
A tool to replace all words with antonyms
First amendment and employment: Can a police department terminate an officer for speech?
Russian equivalents of "We would never hear the end of it"
Bitcoin successfully deducted on sender wallet but did not reach receiver wallet
how to differentiate when a child lwc component is called twice in parent component?
Should I ask for permission to write an expository post about someone's else research?
What happen to those who died but not from the snap?
Why do oscilloscopes use SMPSes instead of linear power supplies?
How can you evade tax by getting employment income just in equity, then using this equity as collateral to take out loan?
Why isn’t SHA-3 in wider use?
Withdrew when Jimmy met up with Heath
What gave Harry Potter the idea of writing in Tom Riddle's diary?
The cat ate your input again!
Christian apologetics regarding the killing of innocent children during the Genesis flood
How many different ways are there to checkmate in the early game?
What are the uses and limitations of Persuasion, Insight, and Deception against other PCs?
During the Space Shuttle Columbia Disaster of 2003, Why Did The Flight Director Say, "Lock the doors."?
Should you play baroque pieces a semitone lower?
Visa National - No Exit Stamp From France on Return to the UK
Simple Stop watch which i want to extend
AsyncDictionary - Can you break thread safety?
What does "sardine box" mean?
Aggregate $lookup with C#
MongoDb - Joining ObjectId references in the list with related collectionsUnable to join two collection where condition exist in subcollectionMongo linq query left outer joinAggregate $lookup Total size of documents in matching pipeline exceeds maximum document sizeHow to $lookup with MongoDB C# driver?Modeling mongodb stored referenced relationships in .net coreSystem.ArgumentException with C# MongoDB.DriverHow do I calculate someone's age in C#?What is the difference between String and string in C#?Hidden Features of C#?Cast int to enum in C#How do you give a C# Auto-Property a default value?How do I enumerate an enum in C#?What are the correct version numbers for C#?How do I get a consistent byte representation of strings in C# without manually specifying an encoding?Get int value from enum in C#How to join two collections using lookup operator in C# Mongodb strongly typed driver
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
I have the following MongoDb query working:
db.Entity.aggregate(
[
"$match":"Id": "12345"
,
"$lookup":
"from": "OtherCollection",
"localField": "otherCollectionId",
"foreignField": "Id",
"as": "ent"
,
"$project":
"Name": 1,
"Date": 1,
"OtherObject": "$arrayElemAt": [ "$ent", 0 ]
,
"$sort":
"OtherObject.Profile.Name": 1
]
)
This retrieves a list of objects joined with a matching object from another collection.
Does anybody know how I can use this in C# using either LINQ or by using this exact string?
I tried using the following code but it can't seem to find the types for QueryDocument
and MongoCursor
- I think they've been deprecated?
BsonDocument document = MongoDB.Bson.Serialization.BsonSerializer.Deserialize<BsonDocument>(" name : value ");
QueryDocument queryDoc = new QueryDocument(document);
MongoCursor toReturn = _connectionCollection.Find(queryDoc);
c# mongodb aggregation-framework mongodb-.net-driver
add a comment |
I have the following MongoDb query working:
db.Entity.aggregate(
[
"$match":"Id": "12345"
,
"$lookup":
"from": "OtherCollection",
"localField": "otherCollectionId",
"foreignField": "Id",
"as": "ent"
,
"$project":
"Name": 1,
"Date": 1,
"OtherObject": "$arrayElemAt": [ "$ent", 0 ]
,
"$sort":
"OtherObject.Profile.Name": 1
]
)
This retrieves a list of objects joined with a matching object from another collection.
Does anybody know how I can use this in C# using either LINQ or by using this exact string?
I tried using the following code but it can't seem to find the types for QueryDocument
and MongoCursor
- I think they've been deprecated?
BsonDocument document = MongoDB.Bson.Serialization.BsonSerializer.Deserialize<BsonDocument>(" name : value ");
QueryDocument queryDoc = new QueryDocument(document);
MongoCursor toReturn = _connectionCollection.Find(queryDoc);
c# mongodb aggregation-framework mongodb-.net-driver
have a look at the mongodb driver wrapper library MongoDB.Entities. it come with built-in relationship management and linq support which trnaslates into aggregate queries without much effort.
– Ryan Gunner
May 6 at 14:27
add a comment |
I have the following MongoDb query working:
db.Entity.aggregate(
[
"$match":"Id": "12345"
,
"$lookup":
"from": "OtherCollection",
"localField": "otherCollectionId",
"foreignField": "Id",
"as": "ent"
,
"$project":
"Name": 1,
"Date": 1,
"OtherObject": "$arrayElemAt": [ "$ent", 0 ]
,
"$sort":
"OtherObject.Profile.Name": 1
]
)
This retrieves a list of objects joined with a matching object from another collection.
Does anybody know how I can use this in C# using either LINQ or by using this exact string?
I tried using the following code but it can't seem to find the types for QueryDocument
and MongoCursor
- I think they've been deprecated?
BsonDocument document = MongoDB.Bson.Serialization.BsonSerializer.Deserialize<BsonDocument>(" name : value ");
QueryDocument queryDoc = new QueryDocument(document);
MongoCursor toReturn = _connectionCollection.Find(queryDoc);
c# mongodb aggregation-framework mongodb-.net-driver
I have the following MongoDb query working:
db.Entity.aggregate(
[
"$match":"Id": "12345"
,
"$lookup":
"from": "OtherCollection",
"localField": "otherCollectionId",
"foreignField": "Id",
"as": "ent"
,
"$project":
"Name": 1,
"Date": 1,
"OtherObject": "$arrayElemAt": [ "$ent", 0 ]
,
"$sort":
"OtherObject.Profile.Name": 1
]
)
This retrieves a list of objects joined with a matching object from another collection.
Does anybody know how I can use this in C# using either LINQ or by using this exact string?
I tried using the following code but it can't seem to find the types for QueryDocument
and MongoCursor
- I think they've been deprecated?
BsonDocument document = MongoDB.Bson.Serialization.BsonSerializer.Deserialize<BsonDocument>(" name : value ");
QueryDocument queryDoc = new QueryDocument(document);
MongoCursor toReturn = _connectionCollection.Find(queryDoc);
c# mongodb aggregation-framework mongodb-.net-driver
c# mongodb aggregation-framework mongodb-.net-driver
edited May 26 '18 at 6:39
Neil Lunn
106k24 gold badges195 silver badges199 bronze badges
106k24 gold badges195 silver badges199 bronze badges
asked May 25 '18 at 13:29
TomSelleckTomSelleck
3,13614 gold badges57 silver badges116 bronze badges
3,13614 gold badges57 silver badges116 bronze badges
have a look at the mongodb driver wrapper library MongoDB.Entities. it come with built-in relationship management and linq support which trnaslates into aggregate queries without much effort.
– Ryan Gunner
May 6 at 14:27
add a comment |
have a look at the mongodb driver wrapper library MongoDB.Entities. it come with built-in relationship management and linq support which trnaslates into aggregate queries without much effort.
– Ryan Gunner
May 6 at 14:27
have a look at the mongodb driver wrapper library MongoDB.Entities. it come with built-in relationship management and linq support which trnaslates into aggregate queries without much effort.
– Ryan Gunner
May 6 at 14:27
have a look at the mongodb driver wrapper library MongoDB.Entities. it come with built-in relationship management and linq support which trnaslates into aggregate queries without much effort.
– Ryan Gunner
May 6 at 14:27
add a comment |
2 Answers
2
active
oldest
votes
There is no need to parse the JSON. Everything here can actually be done directly with either LINQ or the Aggregate Fluent interfaces.
Just using some demonstration classes because the question does not really give much to go on.
Setup
Basically we have two collections here, being
entities
"_id" : ObjectId("5b08ceb40a8a7614c70a5710"), "name" : "A"
"_id" : ObjectId("5b08ceb40a8a7614c70a5711"), "name" : "B"
and others
"_id" : ObjectId("5b08cef10a8a7614c70a5712"),
"entity" : ObjectId("5b08ceb40a8a7614c70a5710"),
"name" : "Sub-A"
"_id" : ObjectId("5b08cefd0a8a7614c70a5713"),
"entity" : ObjectId("5b08ceb40a8a7614c70a5711"),
"name" : "Sub-B"
And a couple of classes to bind them to, just as very basic examples:
public class Entity
public ObjectId id;
public string name get; set;
public class Other
public ObjectId id;
public ObjectId entity get; set;
public string name get; set;
public class EntityWithOthers
public ObjectId id;
public string name get; set;
public IEnumerable<Other> others;
public class EntityWithOther
public ObjectId id;
public string name get; set;
public Other others;
Queries
Fluent Interface
var listNames = new[] "A", "B" ;
var query = entities.Aggregate()
.Match(p => listNames.Contains(p.name))
.Lookup(
foreignCollection: others,
localField: e => e.id,
foreignField: f => f.entity,
@as: (EntityWithOthers eo) => eo.others
)
.Project(p => new p.id, p.name, other = p.others.First() )
.Sort(new BsonDocument("other.name",-1))
.ToList();
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "others"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$arrayElemAt" : [ "$others", 0 ] ,
"_id" : 0
,
"$sort" : "other.name" : -1
]
Probably the easiest to understand since the fluent interface is basically the same as the general BSON structure. The $lookup
stage has all the same arguments and the $arrayElemAt
is represented with First()
. For the $sort
you can simply supply a BSON document or other valid expression.
An alternate is the newer expressive form of $lookup
with a sub-pipeline statement for MongoDB 3.6 and above.
BsonArray subpipeline = new BsonArray();
subpipeline.Add(
new BsonDocument("$match",new BsonDocument(
"$expr", new BsonDocument(
"$eq", new BsonArray "$$entity", "$entity"
)
))
);
var lookup = new BsonDocument("$lookup",
new BsonDocument("from", "others")
.Add("let", new BsonDocument("entity", "$_id"))
.Add("pipeline", subpipeline)
.Add("as","others")
);
var query = entities.Aggregate()
.Match(p => listNames.Contains(p.name))
.AppendStage<EntityWithOthers>(lookup)
.Unwind<EntityWithOthers, EntityWithOther>(p => p.others)
.SortByDescending(p => p.others.name)
.ToList();
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"let" : "entity" : "$_id" ,
"pipeline" : [
"$match" : "$expr" : "$eq" : [ "$$entity", "$entity" ]
],
"as" : "others"
,
"$unwind" : "$others" ,
"$sort" : "others.name" : -1
]
The Fluent "Builder" does not support the syntax directly yet, nor do LINQ Expressions support the $expr
operator, however you can still construct using BsonDocument
and BsonArray
or other valid expressions. Here we also "type" the $unwind
result in order to apply a $sort
using an expression rather than a BsonDocument
as shown earlier.
Aside from other uses, a primary task of a "sub-pipeline" is to reduce the documents returned in the target array of $lookup
. Also the $unwind
here serves a purpose of actually being "merged" into the $lookup
statement on server execution, so this is typically more efficient than just grabbing the first element of the resulting array.
Queryable GroupJoin
var query = entities.AsQueryable()
.Where(p => listNames.Contains(p.name))
.GroupJoin(
others.AsQueryable(),
p => p.id,
o => o.entity,
(p, o) => new p.id, p.name, other = o.First()
)
.OrderByDescending(p => p.other.name);
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "o"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$arrayElemAt" : [ "$o", 0 ] ,
"_id" : 0
,
"$sort" : "other.name" : -1
]
This is almost identical but just using the different interface and produces a slightly different BSON statement, and really only because of the simplified naming in the functional statements. This does bring up the other possibility of simply using an $unwind
as produced from a SelectMany()
:
var query = entities.AsQueryable()
.Where(p => listNames.Contains(p.name))
.GroupJoin(
others.AsQueryable(),
p => p.id,
o => o.entity,
(p, o) => new p.id, p.name, other = o
)
.SelectMany(p => p.other, (p, other) => new p.id, p.name, other )
.OrderByDescending(p => p.other.name);
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "o"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$o",
"_id" : 0
,
"$unwind" : "$other" ,
"$project" :
"id" : "$id",
"name" : "$name",
"other" : "$other",
"_id" : 0
,
"$sort" : "other.name" : -1
]
Normally placing an $unwind
directly following $lookup
is actually an "optimized pattern" for the aggregation framework. However the .NET driver does mess this up in this combination by forcing a $project
in between rather than using the implied naming on the "as"
. If not for that, this is actually better than the $arrayElemAt
when you know you have "one" related result. If you want the $unwind
"coalescence", then you are better off using the fluent interface, or a different form as demonstrated later.
Querable Natural
var query = from p in entities.AsQueryable()
where listNames.Contains(p.name)
join o in others.AsQueryable() on p.id equals o.entity into joined
select new p.id, p.name, other = joined.First()
into p
orderby p.other.name descending
select p;
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "joined"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$arrayElemAt" : [ "$joined", 0 ] ,
"_id" : 0
,
"$sort" : "other.name" : -1
]
All pretty familiar and really just down to functional naming. Just as with using the $unwind
option:
var query = from p in entities.AsQueryable()
where listNames.Contains(p.name)
join o in others.AsQueryable() on p.id equals o.entity into joined
from sub_o in joined.DefaultIfEmpty()
select new p.id, p.name, other = sub_o
into p
orderby p.other.name descending
select p;
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "joined"
,
"$unwind" :
"path" : "$joined", "preserveNullAndEmptyArrays" : true
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$joined",
"_id" : 0
,
"$sort" : "other.name" : -1
]
Which actually is using the "optimized coalescence" form. The translator still insists on adding a $project
since we need the intermediate select
in order to make the statement valid.
Summary
So there are quite a few ways to essentially arrive at what is basically the same query statement with exactly the same results. Whilst you "could" parse the JSON to BsonDocument
form and feed this to the fluent Aggregate()
command, it's generally better to use the natural builders or the LINQ interfaces as they do easily map onto the same statement.
The options with $unwind
are largely shown because even with a "singular" match that "coalescence" form is actually far more optimal then using $arrayElemAt
to take the "first" array element. This even becomes more important with considerations of things like the BSON Limit where the $lookup
target array could cause the parent document to exceed 16MB without further filtering. There is another post here on Aggregate $lookup Total size of documents in matching pipeline exceeds maximum document size where I actually discuss how to avoid that limit being hit by using such options or other Lookup()
syntax available to the fluent interface only at this time.
1
Really great answer - thanks for taking the time to do this.
– TomSelleck
May 28 '18 at 13:01
add a comment |
here's how to do it with MongoDB.Entities. in cases where two entities are in either a one-to-many or many-to-many relationship, you can get reverse relationship access without having to do joins manually as shown below. [disclaimer: i'm the author of the library]
using System;
using System.Linq;
using MongoDB.Entities;
using MongoDB.Driver.Linq;
namespace StackOverflow
public class Program
public class Author : Entity
public string Name get; set;
public Many<Book> Books get; set;
public Author() => this.InitOneToMany(() => Books);
public class Book : Entity
public string Title get; set;
static void Main(string[] args)
new DB("test");
var book = new Book Title = "The Power Of Now" ;
book.Save();
var author = new Author Name = "Eckhart Tolle" ;
author.Save();
author.Books.Add(book);
//build a query for finding all books that has Power in the title.
var bookQuery = DB.Queryable<Book>()
.Where(b => b.Title.Contains("Power"));
//find all the authors of books that has a title with Power in them
var authors = author.Books
.ParentsQueryable<Author>(bookQuery); //also can pass in an ID or array of IDs
//get the result
var result = authors.ToArray();
//output the aggregation pipeline
Console.WriteLine(authors.ToString());
Console.ReadKey();
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f50530363%2faggregate-lookup-with-c-sharp%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
There is no need to parse the JSON. Everything here can actually be done directly with either LINQ or the Aggregate Fluent interfaces.
Just using some demonstration classes because the question does not really give much to go on.
Setup
Basically we have two collections here, being
entities
"_id" : ObjectId("5b08ceb40a8a7614c70a5710"), "name" : "A"
"_id" : ObjectId("5b08ceb40a8a7614c70a5711"), "name" : "B"
and others
"_id" : ObjectId("5b08cef10a8a7614c70a5712"),
"entity" : ObjectId("5b08ceb40a8a7614c70a5710"),
"name" : "Sub-A"
"_id" : ObjectId("5b08cefd0a8a7614c70a5713"),
"entity" : ObjectId("5b08ceb40a8a7614c70a5711"),
"name" : "Sub-B"
And a couple of classes to bind them to, just as very basic examples:
public class Entity
public ObjectId id;
public string name get; set;
public class Other
public ObjectId id;
public ObjectId entity get; set;
public string name get; set;
public class EntityWithOthers
public ObjectId id;
public string name get; set;
public IEnumerable<Other> others;
public class EntityWithOther
public ObjectId id;
public string name get; set;
public Other others;
Queries
Fluent Interface
var listNames = new[] "A", "B" ;
var query = entities.Aggregate()
.Match(p => listNames.Contains(p.name))
.Lookup(
foreignCollection: others,
localField: e => e.id,
foreignField: f => f.entity,
@as: (EntityWithOthers eo) => eo.others
)
.Project(p => new p.id, p.name, other = p.others.First() )
.Sort(new BsonDocument("other.name",-1))
.ToList();
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "others"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$arrayElemAt" : [ "$others", 0 ] ,
"_id" : 0
,
"$sort" : "other.name" : -1
]
Probably the easiest to understand since the fluent interface is basically the same as the general BSON structure. The $lookup
stage has all the same arguments and the $arrayElemAt
is represented with First()
. For the $sort
you can simply supply a BSON document or other valid expression.
An alternate is the newer expressive form of $lookup
with a sub-pipeline statement for MongoDB 3.6 and above.
BsonArray subpipeline = new BsonArray();
subpipeline.Add(
new BsonDocument("$match",new BsonDocument(
"$expr", new BsonDocument(
"$eq", new BsonArray "$$entity", "$entity"
)
))
);
var lookup = new BsonDocument("$lookup",
new BsonDocument("from", "others")
.Add("let", new BsonDocument("entity", "$_id"))
.Add("pipeline", subpipeline)
.Add("as","others")
);
var query = entities.Aggregate()
.Match(p => listNames.Contains(p.name))
.AppendStage<EntityWithOthers>(lookup)
.Unwind<EntityWithOthers, EntityWithOther>(p => p.others)
.SortByDescending(p => p.others.name)
.ToList();
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"let" : "entity" : "$_id" ,
"pipeline" : [
"$match" : "$expr" : "$eq" : [ "$$entity", "$entity" ]
],
"as" : "others"
,
"$unwind" : "$others" ,
"$sort" : "others.name" : -1
]
The Fluent "Builder" does not support the syntax directly yet, nor do LINQ Expressions support the $expr
operator, however you can still construct using BsonDocument
and BsonArray
or other valid expressions. Here we also "type" the $unwind
result in order to apply a $sort
using an expression rather than a BsonDocument
as shown earlier.
Aside from other uses, a primary task of a "sub-pipeline" is to reduce the documents returned in the target array of $lookup
. Also the $unwind
here serves a purpose of actually being "merged" into the $lookup
statement on server execution, so this is typically more efficient than just grabbing the first element of the resulting array.
Queryable GroupJoin
var query = entities.AsQueryable()
.Where(p => listNames.Contains(p.name))
.GroupJoin(
others.AsQueryable(),
p => p.id,
o => o.entity,
(p, o) => new p.id, p.name, other = o.First()
)
.OrderByDescending(p => p.other.name);
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "o"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$arrayElemAt" : [ "$o", 0 ] ,
"_id" : 0
,
"$sort" : "other.name" : -1
]
This is almost identical but just using the different interface and produces a slightly different BSON statement, and really only because of the simplified naming in the functional statements. This does bring up the other possibility of simply using an $unwind
as produced from a SelectMany()
:
var query = entities.AsQueryable()
.Where(p => listNames.Contains(p.name))
.GroupJoin(
others.AsQueryable(),
p => p.id,
o => o.entity,
(p, o) => new p.id, p.name, other = o
)
.SelectMany(p => p.other, (p, other) => new p.id, p.name, other )
.OrderByDescending(p => p.other.name);
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "o"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$o",
"_id" : 0
,
"$unwind" : "$other" ,
"$project" :
"id" : "$id",
"name" : "$name",
"other" : "$other",
"_id" : 0
,
"$sort" : "other.name" : -1
]
Normally placing an $unwind
directly following $lookup
is actually an "optimized pattern" for the aggregation framework. However the .NET driver does mess this up in this combination by forcing a $project
in between rather than using the implied naming on the "as"
. If not for that, this is actually better than the $arrayElemAt
when you know you have "one" related result. If you want the $unwind
"coalescence", then you are better off using the fluent interface, or a different form as demonstrated later.
Querable Natural
var query = from p in entities.AsQueryable()
where listNames.Contains(p.name)
join o in others.AsQueryable() on p.id equals o.entity into joined
select new p.id, p.name, other = joined.First()
into p
orderby p.other.name descending
select p;
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "joined"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$arrayElemAt" : [ "$joined", 0 ] ,
"_id" : 0
,
"$sort" : "other.name" : -1
]
All pretty familiar and really just down to functional naming. Just as with using the $unwind
option:
var query = from p in entities.AsQueryable()
where listNames.Contains(p.name)
join o in others.AsQueryable() on p.id equals o.entity into joined
from sub_o in joined.DefaultIfEmpty()
select new p.id, p.name, other = sub_o
into p
orderby p.other.name descending
select p;
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "joined"
,
"$unwind" :
"path" : "$joined", "preserveNullAndEmptyArrays" : true
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$joined",
"_id" : 0
,
"$sort" : "other.name" : -1
]
Which actually is using the "optimized coalescence" form. The translator still insists on adding a $project
since we need the intermediate select
in order to make the statement valid.
Summary
So there are quite a few ways to essentially arrive at what is basically the same query statement with exactly the same results. Whilst you "could" parse the JSON to BsonDocument
form and feed this to the fluent Aggregate()
command, it's generally better to use the natural builders or the LINQ interfaces as they do easily map onto the same statement.
The options with $unwind
are largely shown because even with a "singular" match that "coalescence" form is actually far more optimal then using $arrayElemAt
to take the "first" array element. This even becomes more important with considerations of things like the BSON Limit where the $lookup
target array could cause the parent document to exceed 16MB without further filtering. There is another post here on Aggregate $lookup Total size of documents in matching pipeline exceeds maximum document size where I actually discuss how to avoid that limit being hit by using such options or other Lookup()
syntax available to the fluent interface only at this time.
1
Really great answer - thanks for taking the time to do this.
– TomSelleck
May 28 '18 at 13:01
add a comment |
There is no need to parse the JSON. Everything here can actually be done directly with either LINQ or the Aggregate Fluent interfaces.
Just using some demonstration classes because the question does not really give much to go on.
Setup
Basically we have two collections here, being
entities
"_id" : ObjectId("5b08ceb40a8a7614c70a5710"), "name" : "A"
"_id" : ObjectId("5b08ceb40a8a7614c70a5711"), "name" : "B"
and others
"_id" : ObjectId("5b08cef10a8a7614c70a5712"),
"entity" : ObjectId("5b08ceb40a8a7614c70a5710"),
"name" : "Sub-A"
"_id" : ObjectId("5b08cefd0a8a7614c70a5713"),
"entity" : ObjectId("5b08ceb40a8a7614c70a5711"),
"name" : "Sub-B"
And a couple of classes to bind them to, just as very basic examples:
public class Entity
public ObjectId id;
public string name get; set;
public class Other
public ObjectId id;
public ObjectId entity get; set;
public string name get; set;
public class EntityWithOthers
public ObjectId id;
public string name get; set;
public IEnumerable<Other> others;
public class EntityWithOther
public ObjectId id;
public string name get; set;
public Other others;
Queries
Fluent Interface
var listNames = new[] "A", "B" ;
var query = entities.Aggregate()
.Match(p => listNames.Contains(p.name))
.Lookup(
foreignCollection: others,
localField: e => e.id,
foreignField: f => f.entity,
@as: (EntityWithOthers eo) => eo.others
)
.Project(p => new p.id, p.name, other = p.others.First() )
.Sort(new BsonDocument("other.name",-1))
.ToList();
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "others"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$arrayElemAt" : [ "$others", 0 ] ,
"_id" : 0
,
"$sort" : "other.name" : -1
]
Probably the easiest to understand since the fluent interface is basically the same as the general BSON structure. The $lookup
stage has all the same arguments and the $arrayElemAt
is represented with First()
. For the $sort
you can simply supply a BSON document or other valid expression.
An alternate is the newer expressive form of $lookup
with a sub-pipeline statement for MongoDB 3.6 and above.
BsonArray subpipeline = new BsonArray();
subpipeline.Add(
new BsonDocument("$match",new BsonDocument(
"$expr", new BsonDocument(
"$eq", new BsonArray "$$entity", "$entity"
)
))
);
var lookup = new BsonDocument("$lookup",
new BsonDocument("from", "others")
.Add("let", new BsonDocument("entity", "$_id"))
.Add("pipeline", subpipeline)
.Add("as","others")
);
var query = entities.Aggregate()
.Match(p => listNames.Contains(p.name))
.AppendStage<EntityWithOthers>(lookup)
.Unwind<EntityWithOthers, EntityWithOther>(p => p.others)
.SortByDescending(p => p.others.name)
.ToList();
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"let" : "entity" : "$_id" ,
"pipeline" : [
"$match" : "$expr" : "$eq" : [ "$$entity", "$entity" ]
],
"as" : "others"
,
"$unwind" : "$others" ,
"$sort" : "others.name" : -1
]
The Fluent "Builder" does not support the syntax directly yet, nor do LINQ Expressions support the $expr
operator, however you can still construct using BsonDocument
and BsonArray
or other valid expressions. Here we also "type" the $unwind
result in order to apply a $sort
using an expression rather than a BsonDocument
as shown earlier.
Aside from other uses, a primary task of a "sub-pipeline" is to reduce the documents returned in the target array of $lookup
. Also the $unwind
here serves a purpose of actually being "merged" into the $lookup
statement on server execution, so this is typically more efficient than just grabbing the first element of the resulting array.
Queryable GroupJoin
var query = entities.AsQueryable()
.Where(p => listNames.Contains(p.name))
.GroupJoin(
others.AsQueryable(),
p => p.id,
o => o.entity,
(p, o) => new p.id, p.name, other = o.First()
)
.OrderByDescending(p => p.other.name);
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "o"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$arrayElemAt" : [ "$o", 0 ] ,
"_id" : 0
,
"$sort" : "other.name" : -1
]
This is almost identical but just using the different interface and produces a slightly different BSON statement, and really only because of the simplified naming in the functional statements. This does bring up the other possibility of simply using an $unwind
as produced from a SelectMany()
:
var query = entities.AsQueryable()
.Where(p => listNames.Contains(p.name))
.GroupJoin(
others.AsQueryable(),
p => p.id,
o => o.entity,
(p, o) => new p.id, p.name, other = o
)
.SelectMany(p => p.other, (p, other) => new p.id, p.name, other )
.OrderByDescending(p => p.other.name);
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "o"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$o",
"_id" : 0
,
"$unwind" : "$other" ,
"$project" :
"id" : "$id",
"name" : "$name",
"other" : "$other",
"_id" : 0
,
"$sort" : "other.name" : -1
]
Normally placing an $unwind
directly following $lookup
is actually an "optimized pattern" for the aggregation framework. However the .NET driver does mess this up in this combination by forcing a $project
in between rather than using the implied naming on the "as"
. If not for that, this is actually better than the $arrayElemAt
when you know you have "one" related result. If you want the $unwind
"coalescence", then you are better off using the fluent interface, or a different form as demonstrated later.
Querable Natural
var query = from p in entities.AsQueryable()
where listNames.Contains(p.name)
join o in others.AsQueryable() on p.id equals o.entity into joined
select new p.id, p.name, other = joined.First()
into p
orderby p.other.name descending
select p;
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "joined"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$arrayElemAt" : [ "$joined", 0 ] ,
"_id" : 0
,
"$sort" : "other.name" : -1
]
All pretty familiar and really just down to functional naming. Just as with using the $unwind
option:
var query = from p in entities.AsQueryable()
where listNames.Contains(p.name)
join o in others.AsQueryable() on p.id equals o.entity into joined
from sub_o in joined.DefaultIfEmpty()
select new p.id, p.name, other = sub_o
into p
orderby p.other.name descending
select p;
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "joined"
,
"$unwind" :
"path" : "$joined", "preserveNullAndEmptyArrays" : true
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$joined",
"_id" : 0
,
"$sort" : "other.name" : -1
]
Which actually is using the "optimized coalescence" form. The translator still insists on adding a $project
since we need the intermediate select
in order to make the statement valid.
Summary
So there are quite a few ways to essentially arrive at what is basically the same query statement with exactly the same results. Whilst you "could" parse the JSON to BsonDocument
form and feed this to the fluent Aggregate()
command, it's generally better to use the natural builders or the LINQ interfaces as they do easily map onto the same statement.
The options with $unwind
are largely shown because even with a "singular" match that "coalescence" form is actually far more optimal then using $arrayElemAt
to take the "first" array element. This even becomes more important with considerations of things like the BSON Limit where the $lookup
target array could cause the parent document to exceed 16MB without further filtering. There is another post here on Aggregate $lookup Total size of documents in matching pipeline exceeds maximum document size where I actually discuss how to avoid that limit being hit by using such options or other Lookup()
syntax available to the fluent interface only at this time.
1
Really great answer - thanks for taking the time to do this.
– TomSelleck
May 28 '18 at 13:01
add a comment |
There is no need to parse the JSON. Everything here can actually be done directly with either LINQ or the Aggregate Fluent interfaces.
Just using some demonstration classes because the question does not really give much to go on.
Setup
Basically we have two collections here, being
entities
"_id" : ObjectId("5b08ceb40a8a7614c70a5710"), "name" : "A"
"_id" : ObjectId("5b08ceb40a8a7614c70a5711"), "name" : "B"
and others
"_id" : ObjectId("5b08cef10a8a7614c70a5712"),
"entity" : ObjectId("5b08ceb40a8a7614c70a5710"),
"name" : "Sub-A"
"_id" : ObjectId("5b08cefd0a8a7614c70a5713"),
"entity" : ObjectId("5b08ceb40a8a7614c70a5711"),
"name" : "Sub-B"
And a couple of classes to bind them to, just as very basic examples:
public class Entity
public ObjectId id;
public string name get; set;
public class Other
public ObjectId id;
public ObjectId entity get; set;
public string name get; set;
public class EntityWithOthers
public ObjectId id;
public string name get; set;
public IEnumerable<Other> others;
public class EntityWithOther
public ObjectId id;
public string name get; set;
public Other others;
Queries
Fluent Interface
var listNames = new[] "A", "B" ;
var query = entities.Aggregate()
.Match(p => listNames.Contains(p.name))
.Lookup(
foreignCollection: others,
localField: e => e.id,
foreignField: f => f.entity,
@as: (EntityWithOthers eo) => eo.others
)
.Project(p => new p.id, p.name, other = p.others.First() )
.Sort(new BsonDocument("other.name",-1))
.ToList();
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "others"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$arrayElemAt" : [ "$others", 0 ] ,
"_id" : 0
,
"$sort" : "other.name" : -1
]
Probably the easiest to understand since the fluent interface is basically the same as the general BSON structure. The $lookup
stage has all the same arguments and the $arrayElemAt
is represented with First()
. For the $sort
you can simply supply a BSON document or other valid expression.
An alternate is the newer expressive form of $lookup
with a sub-pipeline statement for MongoDB 3.6 and above.
BsonArray subpipeline = new BsonArray();
subpipeline.Add(
new BsonDocument("$match",new BsonDocument(
"$expr", new BsonDocument(
"$eq", new BsonArray "$$entity", "$entity"
)
))
);
var lookup = new BsonDocument("$lookup",
new BsonDocument("from", "others")
.Add("let", new BsonDocument("entity", "$_id"))
.Add("pipeline", subpipeline)
.Add("as","others")
);
var query = entities.Aggregate()
.Match(p => listNames.Contains(p.name))
.AppendStage<EntityWithOthers>(lookup)
.Unwind<EntityWithOthers, EntityWithOther>(p => p.others)
.SortByDescending(p => p.others.name)
.ToList();
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"let" : "entity" : "$_id" ,
"pipeline" : [
"$match" : "$expr" : "$eq" : [ "$$entity", "$entity" ]
],
"as" : "others"
,
"$unwind" : "$others" ,
"$sort" : "others.name" : -1
]
The Fluent "Builder" does not support the syntax directly yet, nor do LINQ Expressions support the $expr
operator, however you can still construct using BsonDocument
and BsonArray
or other valid expressions. Here we also "type" the $unwind
result in order to apply a $sort
using an expression rather than a BsonDocument
as shown earlier.
Aside from other uses, a primary task of a "sub-pipeline" is to reduce the documents returned in the target array of $lookup
. Also the $unwind
here serves a purpose of actually being "merged" into the $lookup
statement on server execution, so this is typically more efficient than just grabbing the first element of the resulting array.
Queryable GroupJoin
var query = entities.AsQueryable()
.Where(p => listNames.Contains(p.name))
.GroupJoin(
others.AsQueryable(),
p => p.id,
o => o.entity,
(p, o) => new p.id, p.name, other = o.First()
)
.OrderByDescending(p => p.other.name);
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "o"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$arrayElemAt" : [ "$o", 0 ] ,
"_id" : 0
,
"$sort" : "other.name" : -1
]
This is almost identical but just using the different interface and produces a slightly different BSON statement, and really only because of the simplified naming in the functional statements. This does bring up the other possibility of simply using an $unwind
as produced from a SelectMany()
:
var query = entities.AsQueryable()
.Where(p => listNames.Contains(p.name))
.GroupJoin(
others.AsQueryable(),
p => p.id,
o => o.entity,
(p, o) => new p.id, p.name, other = o
)
.SelectMany(p => p.other, (p, other) => new p.id, p.name, other )
.OrderByDescending(p => p.other.name);
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "o"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$o",
"_id" : 0
,
"$unwind" : "$other" ,
"$project" :
"id" : "$id",
"name" : "$name",
"other" : "$other",
"_id" : 0
,
"$sort" : "other.name" : -1
]
Normally placing an $unwind
directly following $lookup
is actually an "optimized pattern" for the aggregation framework. However the .NET driver does mess this up in this combination by forcing a $project
in between rather than using the implied naming on the "as"
. If not for that, this is actually better than the $arrayElemAt
when you know you have "one" related result. If you want the $unwind
"coalescence", then you are better off using the fluent interface, or a different form as demonstrated later.
Querable Natural
var query = from p in entities.AsQueryable()
where listNames.Contains(p.name)
join o in others.AsQueryable() on p.id equals o.entity into joined
select new p.id, p.name, other = joined.First()
into p
orderby p.other.name descending
select p;
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "joined"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$arrayElemAt" : [ "$joined", 0 ] ,
"_id" : 0
,
"$sort" : "other.name" : -1
]
All pretty familiar and really just down to functional naming. Just as with using the $unwind
option:
var query = from p in entities.AsQueryable()
where listNames.Contains(p.name)
join o in others.AsQueryable() on p.id equals o.entity into joined
from sub_o in joined.DefaultIfEmpty()
select new p.id, p.name, other = sub_o
into p
orderby p.other.name descending
select p;
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "joined"
,
"$unwind" :
"path" : "$joined", "preserveNullAndEmptyArrays" : true
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$joined",
"_id" : 0
,
"$sort" : "other.name" : -1
]
Which actually is using the "optimized coalescence" form. The translator still insists on adding a $project
since we need the intermediate select
in order to make the statement valid.
Summary
So there are quite a few ways to essentially arrive at what is basically the same query statement with exactly the same results. Whilst you "could" parse the JSON to BsonDocument
form and feed this to the fluent Aggregate()
command, it's generally better to use the natural builders or the LINQ interfaces as they do easily map onto the same statement.
The options with $unwind
are largely shown because even with a "singular" match that "coalescence" form is actually far more optimal then using $arrayElemAt
to take the "first" array element. This even becomes more important with considerations of things like the BSON Limit where the $lookup
target array could cause the parent document to exceed 16MB without further filtering. There is another post here on Aggregate $lookup Total size of documents in matching pipeline exceeds maximum document size where I actually discuss how to avoid that limit being hit by using such options or other Lookup()
syntax available to the fluent interface only at this time.
There is no need to parse the JSON. Everything here can actually be done directly with either LINQ or the Aggregate Fluent interfaces.
Just using some demonstration classes because the question does not really give much to go on.
Setup
Basically we have two collections here, being
entities
"_id" : ObjectId("5b08ceb40a8a7614c70a5710"), "name" : "A"
"_id" : ObjectId("5b08ceb40a8a7614c70a5711"), "name" : "B"
and others
"_id" : ObjectId("5b08cef10a8a7614c70a5712"),
"entity" : ObjectId("5b08ceb40a8a7614c70a5710"),
"name" : "Sub-A"
"_id" : ObjectId("5b08cefd0a8a7614c70a5713"),
"entity" : ObjectId("5b08ceb40a8a7614c70a5711"),
"name" : "Sub-B"
And a couple of classes to bind them to, just as very basic examples:
public class Entity
public ObjectId id;
public string name get; set;
public class Other
public ObjectId id;
public ObjectId entity get; set;
public string name get; set;
public class EntityWithOthers
public ObjectId id;
public string name get; set;
public IEnumerable<Other> others;
public class EntityWithOther
public ObjectId id;
public string name get; set;
public Other others;
Queries
Fluent Interface
var listNames = new[] "A", "B" ;
var query = entities.Aggregate()
.Match(p => listNames.Contains(p.name))
.Lookup(
foreignCollection: others,
localField: e => e.id,
foreignField: f => f.entity,
@as: (EntityWithOthers eo) => eo.others
)
.Project(p => new p.id, p.name, other = p.others.First() )
.Sort(new BsonDocument("other.name",-1))
.ToList();
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "others"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$arrayElemAt" : [ "$others", 0 ] ,
"_id" : 0
,
"$sort" : "other.name" : -1
]
Probably the easiest to understand since the fluent interface is basically the same as the general BSON structure. The $lookup
stage has all the same arguments and the $arrayElemAt
is represented with First()
. For the $sort
you can simply supply a BSON document or other valid expression.
An alternate is the newer expressive form of $lookup
with a sub-pipeline statement for MongoDB 3.6 and above.
BsonArray subpipeline = new BsonArray();
subpipeline.Add(
new BsonDocument("$match",new BsonDocument(
"$expr", new BsonDocument(
"$eq", new BsonArray "$$entity", "$entity"
)
))
);
var lookup = new BsonDocument("$lookup",
new BsonDocument("from", "others")
.Add("let", new BsonDocument("entity", "$_id"))
.Add("pipeline", subpipeline)
.Add("as","others")
);
var query = entities.Aggregate()
.Match(p => listNames.Contains(p.name))
.AppendStage<EntityWithOthers>(lookup)
.Unwind<EntityWithOthers, EntityWithOther>(p => p.others)
.SortByDescending(p => p.others.name)
.ToList();
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"let" : "entity" : "$_id" ,
"pipeline" : [
"$match" : "$expr" : "$eq" : [ "$$entity", "$entity" ]
],
"as" : "others"
,
"$unwind" : "$others" ,
"$sort" : "others.name" : -1
]
The Fluent "Builder" does not support the syntax directly yet, nor do LINQ Expressions support the $expr
operator, however you can still construct using BsonDocument
and BsonArray
or other valid expressions. Here we also "type" the $unwind
result in order to apply a $sort
using an expression rather than a BsonDocument
as shown earlier.
Aside from other uses, a primary task of a "sub-pipeline" is to reduce the documents returned in the target array of $lookup
. Also the $unwind
here serves a purpose of actually being "merged" into the $lookup
statement on server execution, so this is typically more efficient than just grabbing the first element of the resulting array.
Queryable GroupJoin
var query = entities.AsQueryable()
.Where(p => listNames.Contains(p.name))
.GroupJoin(
others.AsQueryable(),
p => p.id,
o => o.entity,
(p, o) => new p.id, p.name, other = o.First()
)
.OrderByDescending(p => p.other.name);
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "o"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$arrayElemAt" : [ "$o", 0 ] ,
"_id" : 0
,
"$sort" : "other.name" : -1
]
This is almost identical but just using the different interface and produces a slightly different BSON statement, and really only because of the simplified naming in the functional statements. This does bring up the other possibility of simply using an $unwind
as produced from a SelectMany()
:
var query = entities.AsQueryable()
.Where(p => listNames.Contains(p.name))
.GroupJoin(
others.AsQueryable(),
p => p.id,
o => o.entity,
(p, o) => new p.id, p.name, other = o
)
.SelectMany(p => p.other, (p, other) => new p.id, p.name, other )
.OrderByDescending(p => p.other.name);
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "o"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$o",
"_id" : 0
,
"$unwind" : "$other" ,
"$project" :
"id" : "$id",
"name" : "$name",
"other" : "$other",
"_id" : 0
,
"$sort" : "other.name" : -1
]
Normally placing an $unwind
directly following $lookup
is actually an "optimized pattern" for the aggregation framework. However the .NET driver does mess this up in this combination by forcing a $project
in between rather than using the implied naming on the "as"
. If not for that, this is actually better than the $arrayElemAt
when you know you have "one" related result. If you want the $unwind
"coalescence", then you are better off using the fluent interface, or a different form as demonstrated later.
Querable Natural
var query = from p in entities.AsQueryable()
where listNames.Contains(p.name)
join o in others.AsQueryable() on p.id equals o.entity into joined
select new p.id, p.name, other = joined.First()
into p
orderby p.other.name descending
select p;
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "joined"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$arrayElemAt" : [ "$joined", 0 ] ,
"_id" : 0
,
"$sort" : "other.name" : -1
]
All pretty familiar and really just down to functional naming. Just as with using the $unwind
option:
var query = from p in entities.AsQueryable()
where listNames.Contains(p.name)
join o in others.AsQueryable() on p.id equals o.entity into joined
from sub_o in joined.DefaultIfEmpty()
select new p.id, p.name, other = sub_o
into p
orderby p.other.name descending
select p;
Request sent to server:
[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "joined"
,
"$unwind" :
"path" : "$joined", "preserveNullAndEmptyArrays" : true
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$joined",
"_id" : 0
,
"$sort" : "other.name" : -1
]
Which actually is using the "optimized coalescence" form. The translator still insists on adding a $project
since we need the intermediate select
in order to make the statement valid.
Summary
So there are quite a few ways to essentially arrive at what is basically the same query statement with exactly the same results. Whilst you "could" parse the JSON to BsonDocument
form and feed this to the fluent Aggregate()
command, it's generally better to use the natural builders or the LINQ interfaces as they do easily map onto the same statement.
The options with $unwind
are largely shown because even with a "singular" match that "coalescence" form is actually far more optimal then using $arrayElemAt
to take the "first" array element. This even becomes more important with considerations of things like the BSON Limit where the $lookup
target array could cause the parent document to exceed 16MB without further filtering. There is another post here on Aggregate $lookup Total size of documents in matching pipeline exceeds maximum document size where I actually discuss how to avoid that limit being hit by using such options or other Lookup()
syntax available to the fluent interface only at this time.
edited May 27 '18 at 0:41
answered May 26 '18 at 6:38
Neil LunnNeil Lunn
106k24 gold badges195 silver badges199 bronze badges
106k24 gold badges195 silver badges199 bronze badges
1
Really great answer - thanks for taking the time to do this.
– TomSelleck
May 28 '18 at 13:01
add a comment |
1
Really great answer - thanks for taking the time to do this.
– TomSelleck
May 28 '18 at 13:01
1
1
Really great answer - thanks for taking the time to do this.
– TomSelleck
May 28 '18 at 13:01
Really great answer - thanks for taking the time to do this.
– TomSelleck
May 28 '18 at 13:01
add a comment |
here's how to do it with MongoDB.Entities. in cases where two entities are in either a one-to-many or many-to-many relationship, you can get reverse relationship access without having to do joins manually as shown below. [disclaimer: i'm the author of the library]
using System;
using System.Linq;
using MongoDB.Entities;
using MongoDB.Driver.Linq;
namespace StackOverflow
public class Program
public class Author : Entity
public string Name get; set;
public Many<Book> Books get; set;
public Author() => this.InitOneToMany(() => Books);
public class Book : Entity
public string Title get; set;
static void Main(string[] args)
new DB("test");
var book = new Book Title = "The Power Of Now" ;
book.Save();
var author = new Author Name = "Eckhart Tolle" ;
author.Save();
author.Books.Add(book);
//build a query for finding all books that has Power in the title.
var bookQuery = DB.Queryable<Book>()
.Where(b => b.Title.Contains("Power"));
//find all the authors of books that has a title with Power in them
var authors = author.Books
.ParentsQueryable<Author>(bookQuery); //also can pass in an ID or array of IDs
//get the result
var result = authors.ToArray();
//output the aggregation pipeline
Console.WriteLine(authors.ToString());
Console.ReadKey();
add a comment |
here's how to do it with MongoDB.Entities. in cases where two entities are in either a one-to-many or many-to-many relationship, you can get reverse relationship access without having to do joins manually as shown below. [disclaimer: i'm the author of the library]
using System;
using System.Linq;
using MongoDB.Entities;
using MongoDB.Driver.Linq;
namespace StackOverflow
public class Program
public class Author : Entity
public string Name get; set;
public Many<Book> Books get; set;
public Author() => this.InitOneToMany(() => Books);
public class Book : Entity
public string Title get; set;
static void Main(string[] args)
new DB("test");
var book = new Book Title = "The Power Of Now" ;
book.Save();
var author = new Author Name = "Eckhart Tolle" ;
author.Save();
author.Books.Add(book);
//build a query for finding all books that has Power in the title.
var bookQuery = DB.Queryable<Book>()
.Where(b => b.Title.Contains("Power"));
//find all the authors of books that has a title with Power in them
var authors = author.Books
.ParentsQueryable<Author>(bookQuery); //also can pass in an ID or array of IDs
//get the result
var result = authors.ToArray();
//output the aggregation pipeline
Console.WriteLine(authors.ToString());
Console.ReadKey();
add a comment |
here's how to do it with MongoDB.Entities. in cases where two entities are in either a one-to-many or many-to-many relationship, you can get reverse relationship access without having to do joins manually as shown below. [disclaimer: i'm the author of the library]
using System;
using System.Linq;
using MongoDB.Entities;
using MongoDB.Driver.Linq;
namespace StackOverflow
public class Program
public class Author : Entity
public string Name get; set;
public Many<Book> Books get; set;
public Author() => this.InitOneToMany(() => Books);
public class Book : Entity
public string Title get; set;
static void Main(string[] args)
new DB("test");
var book = new Book Title = "The Power Of Now" ;
book.Save();
var author = new Author Name = "Eckhart Tolle" ;
author.Save();
author.Books.Add(book);
//build a query for finding all books that has Power in the title.
var bookQuery = DB.Queryable<Book>()
.Where(b => b.Title.Contains("Power"));
//find all the authors of books that has a title with Power in them
var authors = author.Books
.ParentsQueryable<Author>(bookQuery); //also can pass in an ID or array of IDs
//get the result
var result = authors.ToArray();
//output the aggregation pipeline
Console.WriteLine(authors.ToString());
Console.ReadKey();
here's how to do it with MongoDB.Entities. in cases where two entities are in either a one-to-many or many-to-many relationship, you can get reverse relationship access without having to do joins manually as shown below. [disclaimer: i'm the author of the library]
using System;
using System.Linq;
using MongoDB.Entities;
using MongoDB.Driver.Linq;
namespace StackOverflow
public class Program
public class Author : Entity
public string Name get; set;
public Many<Book> Books get; set;
public Author() => this.InitOneToMany(() => Books);
public class Book : Entity
public string Title get; set;
static void Main(string[] args)
new DB("test");
var book = new Book Title = "The Power Of Now" ;
book.Save();
var author = new Author Name = "Eckhart Tolle" ;
author.Save();
author.Books.Add(book);
//build a query for finding all books that has Power in the title.
var bookQuery = DB.Queryable<Book>()
.Where(b => b.Title.Contains("Power"));
//find all the authors of books that has a title with Power in them
var authors = author.Books
.ParentsQueryable<Author>(bookQuery); //also can pass in an ID or array of IDs
//get the result
var result = authors.ToArray();
//output the aggregation pipeline
Console.WriteLine(authors.ToString());
Console.ReadKey();
answered Jun 4 at 13:12
Ryan GunnerRyan Gunner
8211 gold badge6 silver badges12 bronze badges
8211 gold badge6 silver badges12 bronze badges
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f50530363%2faggregate-lookup-with-c-sharp%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
have a look at the mongodb driver wrapper library MongoDB.Entities. it come with built-in relationship management and linq support which trnaslates into aggregate queries without much effort.
– Ryan Gunner
May 6 at 14:27