Aggregate $lookup with C#MongoDb - Joining ObjectId references in the list with related collectionsUnable to join two collection where condition exist in subcollectionMongo linq query left outer joinAggregate $lookup Total size of documents in matching pipeline exceeds maximum document sizeHow to $lookup with MongoDB C# driver?Modeling mongodb stored referenced relationships in .net coreSystem.ArgumentException with C# MongoDB.DriverHow do I calculate someone's age in C#?What is the difference between String and string in C#?Hidden Features of C#?Cast int to enum in C#How do you give a C# Auto-Property a default value?How do I enumerate an enum in C#?What are the correct version numbers for C#?How do I get a consistent byte representation of strings in C# without manually specifying an encoding?Get int value from enum in C#How to join two collections using lookup operator in C# Mongodb strongly typed driver

Generating function of ordered partitions

Identification of vintage sloping window

A tool to replace all words with antonyms

First amendment and employment: Can a police department terminate an officer for speech?

Russian equivalents of "We would never hear the end of it"

Bitcoin successfully deducted on sender wallet but did not reach receiver wallet

how to differentiate when a child lwc component is called twice in parent component?

Should I ask for permission to write an expository post about someone's else research?

What happen to those who died but not from the snap?

Why do oscilloscopes use SMPSes instead of linear power supplies?

How can you evade tax by getting employment income just in equity, then using this equity as collateral to take out loan?

Why isn’t SHA-3 in wider use?

Withdrew when Jimmy met up with Heath

What gave Harry Potter the idea of writing in Tom Riddle's diary?

The cat ate your input again!

Christian apologetics regarding the killing of innocent children during the Genesis flood

How many different ways are there to checkmate in the early game?

What are the uses and limitations of Persuasion, Insight, and Deception against other PCs?

During the Space Shuttle Columbia Disaster of 2003, Why Did The Flight Director Say, "Lock the doors."?

Should you play baroque pieces a semitone lower?

Visa National - No Exit Stamp From France on Return to the UK

Simple Stop watch which i want to extend

AsyncDictionary - Can you break thread safety?

What does "sardine box" mean?



Aggregate $lookup with C#


MongoDb - Joining ObjectId references in the list with related collectionsUnable to join two collection where condition exist in subcollectionMongo linq query left outer joinAggregate $lookup Total size of documents in matching pipeline exceeds maximum document sizeHow to $lookup with MongoDB C# driver?Modeling mongodb stored referenced relationships in .net coreSystem.ArgumentException with C# MongoDB.DriverHow do I calculate someone's age in C#?What is the difference between String and string in C#?Hidden Features of C#?Cast int to enum in C#How do you give a C# Auto-Property a default value?How do I enumerate an enum in C#?What are the correct version numbers for C#?How do I get a consistent byte representation of strings in C# without manually specifying an encoding?Get int value from enum in C#How to join two collections using lookup operator in C# Mongodb strongly typed driver






.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;








12















I have the following MongoDb query working:



db.Entity.aggregate(
[

"$match":"Id": "12345"
,

"$lookup":
"from": "OtherCollection",
"localField": "otherCollectionId",
"foreignField": "Id",
"as": "ent"

,

"$project":
"Name": 1,
"Date": 1,
"OtherObject": "$arrayElemAt": [ "$ent", 0 ]

,

"$sort":
"OtherObject.Profile.Name": 1


]
)


This retrieves a list of objects joined with a matching object from another collection.



Does anybody know how I can use this in C# using either LINQ or by using this exact string?



I tried using the following code but it can't seem to find the types for QueryDocument and MongoCursor - I think they've been deprecated?



BsonDocument document = MongoDB.Bson.Serialization.BsonSerializer.Deserialize<BsonDocument>(" name : value ");
QueryDocument queryDoc = new QueryDocument(document);
MongoCursor toReturn = _connectionCollection.Find(queryDoc);









share|improve this question


























  • have a look at the mongodb driver wrapper library MongoDB.Entities. it come with built-in relationship management and linq support which trnaslates into aggregate queries without much effort.

    – Ryan Gunner
    May 6 at 14:27

















12















I have the following MongoDb query working:



db.Entity.aggregate(
[

"$match":"Id": "12345"
,

"$lookup":
"from": "OtherCollection",
"localField": "otherCollectionId",
"foreignField": "Id",
"as": "ent"

,

"$project":
"Name": 1,
"Date": 1,
"OtherObject": "$arrayElemAt": [ "$ent", 0 ]

,

"$sort":
"OtherObject.Profile.Name": 1


]
)


This retrieves a list of objects joined with a matching object from another collection.



Does anybody know how I can use this in C# using either LINQ or by using this exact string?



I tried using the following code but it can't seem to find the types for QueryDocument and MongoCursor - I think they've been deprecated?



BsonDocument document = MongoDB.Bson.Serialization.BsonSerializer.Deserialize<BsonDocument>(" name : value ");
QueryDocument queryDoc = new QueryDocument(document);
MongoCursor toReturn = _connectionCollection.Find(queryDoc);









share|improve this question


























  • have a look at the mongodb driver wrapper library MongoDB.Entities. it come with built-in relationship management and linq support which trnaslates into aggregate queries without much effort.

    – Ryan Gunner
    May 6 at 14:27













12












12








12


1






I have the following MongoDb query working:



db.Entity.aggregate(
[

"$match":"Id": "12345"
,

"$lookup":
"from": "OtherCollection",
"localField": "otherCollectionId",
"foreignField": "Id",
"as": "ent"

,

"$project":
"Name": 1,
"Date": 1,
"OtherObject": "$arrayElemAt": [ "$ent", 0 ]

,

"$sort":
"OtherObject.Profile.Name": 1


]
)


This retrieves a list of objects joined with a matching object from another collection.



Does anybody know how I can use this in C# using either LINQ or by using this exact string?



I tried using the following code but it can't seem to find the types for QueryDocument and MongoCursor - I think they've been deprecated?



BsonDocument document = MongoDB.Bson.Serialization.BsonSerializer.Deserialize<BsonDocument>(" name : value ");
QueryDocument queryDoc = new QueryDocument(document);
MongoCursor toReturn = _connectionCollection.Find(queryDoc);









share|improve this question
















I have the following MongoDb query working:



db.Entity.aggregate(
[

"$match":"Id": "12345"
,

"$lookup":
"from": "OtherCollection",
"localField": "otherCollectionId",
"foreignField": "Id",
"as": "ent"

,

"$project":
"Name": 1,
"Date": 1,
"OtherObject": "$arrayElemAt": [ "$ent", 0 ]

,

"$sort":
"OtherObject.Profile.Name": 1


]
)


This retrieves a list of objects joined with a matching object from another collection.



Does anybody know how I can use this in C# using either LINQ or by using this exact string?



I tried using the following code but it can't seem to find the types for QueryDocument and MongoCursor - I think they've been deprecated?



BsonDocument document = MongoDB.Bson.Serialization.BsonSerializer.Deserialize<BsonDocument>(" name : value ");
QueryDocument queryDoc = new QueryDocument(document);
MongoCursor toReturn = _connectionCollection.Find(queryDoc);






c# mongodb aggregation-framework mongodb-.net-driver






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited May 26 '18 at 6:39









Neil Lunn

106k24 gold badges195 silver badges199 bronze badges




106k24 gold badges195 silver badges199 bronze badges










asked May 25 '18 at 13:29









TomSelleckTomSelleck

3,13614 gold badges57 silver badges116 bronze badges




3,13614 gold badges57 silver badges116 bronze badges















  • have a look at the mongodb driver wrapper library MongoDB.Entities. it come with built-in relationship management and linq support which trnaslates into aggregate queries without much effort.

    – Ryan Gunner
    May 6 at 14:27

















  • have a look at the mongodb driver wrapper library MongoDB.Entities. it come with built-in relationship management and linq support which trnaslates into aggregate queries without much effort.

    – Ryan Gunner
    May 6 at 14:27
















have a look at the mongodb driver wrapper library MongoDB.Entities. it come with built-in relationship management and linq support which trnaslates into aggregate queries without much effort.

– Ryan Gunner
May 6 at 14:27





have a look at the mongodb driver wrapper library MongoDB.Entities. it come with built-in relationship management and linq support which trnaslates into aggregate queries without much effort.

– Ryan Gunner
May 6 at 14:27












2 Answers
2






active

oldest

votes


















28














There is no need to parse the JSON. Everything here can actually be done directly with either LINQ or the Aggregate Fluent interfaces.



Just using some demonstration classes because the question does not really give much to go on.



Setup



Basically we have two collections here, being



entities



 "_id" : ObjectId("5b08ceb40a8a7614c70a5710"), "name" : "A" 
"_id" : ObjectId("5b08ceb40a8a7614c70a5711"), "name" : "B"


and others




"_id" : ObjectId("5b08cef10a8a7614c70a5712"),
"entity" : ObjectId("5b08ceb40a8a7614c70a5710"),
"name" : "Sub-A"


"_id" : ObjectId("5b08cefd0a8a7614c70a5713"),
"entity" : ObjectId("5b08ceb40a8a7614c70a5711"),
"name" : "Sub-B"



And a couple of classes to bind them to, just as very basic examples:



public class Entity

public ObjectId id;
public string name get; set;


public class Other

public ObjectId id;
public ObjectId entity get; set;
public string name get; set;


public class EntityWithOthers

public ObjectId id;
public string name get; set;
public IEnumerable<Other> others;


public class EntityWithOther

public ObjectId id;
public string name get; set;
public Other others;



Queries



Fluent Interface



var listNames = new[] "A", "B" ;

var query = entities.Aggregate()
.Match(p => listNames.Contains(p.name))
.Lookup(
foreignCollection: others,
localField: e => e.id,
foreignField: f => f.entity,
@as: (EntityWithOthers eo) => eo.others
)
.Project(p => new p.id, p.name, other = p.others.First() )
.Sort(new BsonDocument("other.name",-1))
.ToList();


Request sent to server:



[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "others"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$arrayElemAt" : [ "$others", 0 ] ,
"_id" : 0
,
"$sort" : "other.name" : -1
]


Probably the easiest to understand since the fluent interface is basically the same as the general BSON structure. The $lookup stage has all the same arguments and the $arrayElemAt is represented with First(). For the $sort you can simply supply a BSON document or other valid expression.



An alternate is the newer expressive form of $lookup with a sub-pipeline statement for MongoDB 3.6 and above.



BsonArray subpipeline = new BsonArray();

subpipeline.Add(
new BsonDocument("$match",new BsonDocument(
"$expr", new BsonDocument(
"$eq", new BsonArray "$$entity", "$entity"
)
))
);

var lookup = new BsonDocument("$lookup",
new BsonDocument("from", "others")
.Add("let", new BsonDocument("entity", "$_id"))
.Add("pipeline", subpipeline)
.Add("as","others")
);

var query = entities.Aggregate()
.Match(p => listNames.Contains(p.name))
.AppendStage<EntityWithOthers>(lookup)
.Unwind<EntityWithOthers, EntityWithOther>(p => p.others)
.SortByDescending(p => p.others.name)
.ToList();


Request sent to server:



[ 
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"let" : "entity" : "$_id" ,
"pipeline" : [
"$match" : "$expr" : "$eq" : [ "$$entity", "$entity" ]
],
"as" : "others"
,
"$unwind" : "$others" ,
"$sort" : "others.name" : -1
]


The Fluent "Builder" does not support the syntax directly yet, nor do LINQ Expressions support the $expr operator, however you can still construct using BsonDocument and BsonArray or other valid expressions. Here we also "type" the $unwind result in order to apply a $sort using an expression rather than a BsonDocument as shown earlier.



Aside from other uses, a primary task of a "sub-pipeline" is to reduce the documents returned in the target array of $lookup. Also the $unwind here serves a purpose of actually being "merged" into the $lookup statement on server execution, so this is typically more efficient than just grabbing the first element of the resulting array.



Queryable GroupJoin



var query = entities.AsQueryable()
.Where(p => listNames.Contains(p.name))
.GroupJoin(
others.AsQueryable(),
p => p.id,
o => o.entity,
(p, o) => new p.id, p.name, other = o.First()
)
.OrderByDescending(p => p.other.name);


Request sent to server:



[ 
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "o"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$arrayElemAt" : [ "$o", 0 ] ,
"_id" : 0
,
"$sort" : "other.name" : -1
]


This is almost identical but just using the different interface and produces a slightly different BSON statement, and really only because of the simplified naming in the functional statements. This does bring up the other possibility of simply using an $unwind as produced from a SelectMany():



var query = entities.AsQueryable()
.Where(p => listNames.Contains(p.name))
.GroupJoin(
others.AsQueryable(),
p => p.id,
o => o.entity,
(p, o) => new p.id, p.name, other = o
)
.SelectMany(p => p.other, (p, other) => new p.id, p.name, other )
.OrderByDescending(p => p.other.name);


Request sent to server:



[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "o"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$o",
"_id" : 0
,
"$unwind" : "$other" ,
"$project" :
"id" : "$id",
"name" : "$name",
"other" : "$other",
"_id" : 0
,
"$sort" : "other.name" : -1
]


Normally placing an $unwind directly following $lookup is actually an "optimized pattern" for the aggregation framework. However the .NET driver does mess this up in this combination by forcing a $project in between rather than using the implied naming on the "as". If not for that, this is actually better than the $arrayElemAt when you know you have "one" related result. If you want the $unwind "coalescence", then you are better off using the fluent interface, or a different form as demonstrated later.



Querable Natural



var query = from p in entities.AsQueryable()
where listNames.Contains(p.name)
join o in others.AsQueryable() on p.id equals o.entity into joined
select new p.id, p.name, other = joined.First()
into p
orderby p.other.name descending
select p;


Request sent to server:



[
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "joined"
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$arrayElemAt" : [ "$joined", 0 ] ,
"_id" : 0
,
"$sort" : "other.name" : -1
]


All pretty familiar and really just down to functional naming. Just as with using the $unwind option:



var query = from p in entities.AsQueryable()
where listNames.Contains(p.name)
join o in others.AsQueryable() on p.id equals o.entity into joined
from sub_o in joined.DefaultIfEmpty()
select new p.id, p.name, other = sub_o
into p
orderby p.other.name descending
select p;


Request sent to server:



[ 
"$match" : "name" : "$in" : [ "A", "B" ] ,
"$lookup" :
"from" : "others",
"localField" : "_id",
"foreignField" : "entity",
"as" : "joined"
,
"$unwind" :
"path" : "$joined", "preserveNullAndEmptyArrays" : true
,
"$project" :
"id" : "$_id",
"name" : "$name",
"other" : "$joined",
"_id" : 0
,
"$sort" : "other.name" : -1
]


Which actually is using the "optimized coalescence" form. The translator still insists on adding a $project since we need the intermediate select in order to make the statement valid.



Summary



So there are quite a few ways to essentially arrive at what is basically the same query statement with exactly the same results. Whilst you "could" parse the JSON to BsonDocument form and feed this to the fluent Aggregate() command, it's generally better to use the natural builders or the LINQ interfaces as they do easily map onto the same statement.



The options with $unwind are largely shown because even with a "singular" match that "coalescence" form is actually far more optimal then using $arrayElemAt to take the "first" array element. This even becomes more important with considerations of things like the BSON Limit where the $lookup target array could cause the parent document to exceed 16MB without further filtering. There is another post here on Aggregate $lookup Total size of documents in matching pipeline exceeds maximum document size where I actually discuss how to avoid that limit being hit by using such options or other Lookup() syntax available to the fluent interface only at this time.






share|improve this answer






















  • 1





    Really great answer - thanks for taking the time to do this.

    – TomSelleck
    May 28 '18 at 13:01


















0














here's how to do it with MongoDB.Entities. in cases where two entities are in either a one-to-many or many-to-many relationship, you can get reverse relationship access without having to do joins manually as shown below. [disclaimer: i'm the author of the library]



using System;
using System.Linq;
using MongoDB.Entities;
using MongoDB.Driver.Linq;

namespace StackOverflow

public class Program

public class Author : Entity

public string Name get; set;
public Many<Book> Books get; set;

public Author() => this.InitOneToMany(() => Books);


public class Book : Entity

public string Title get; set;


static void Main(string[] args)

new DB("test");

var book = new Book Title = "The Power Of Now" ;
book.Save();

var author = new Author Name = "Eckhart Tolle" ;
author.Save();

author.Books.Add(book);

//build a query for finding all books that has Power in the title.
var bookQuery = DB.Queryable<Book>()
.Where(b => b.Title.Contains("Power"));

//find all the authors of books that has a title with Power in them
var authors = author.Books
.ParentsQueryable<Author>(bookQuery); //also can pass in an ID or array of IDs

//get the result
var result = authors.ToArray();

//output the aggregation pipeline
Console.WriteLine(authors.ToString());


Console.ReadKey();








share|improve this answer



























    Your Answer






    StackExchange.ifUsing("editor", function ()
    StackExchange.using("externalEditor", function ()
    StackExchange.using("snippets", function ()
    StackExchange.snippets.init();
    );
    );
    , "code-snippets");

    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "1"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader:
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    ,
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );













    draft saved

    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f50530363%2faggregate-lookup-with-c-sharp%23new-answer', 'question_page');

    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    28














    There is no need to parse the JSON. Everything here can actually be done directly with either LINQ or the Aggregate Fluent interfaces.



    Just using some demonstration classes because the question does not really give much to go on.



    Setup



    Basically we have two collections here, being



    entities



     "_id" : ObjectId("5b08ceb40a8a7614c70a5710"), "name" : "A" 
    "_id" : ObjectId("5b08ceb40a8a7614c70a5711"), "name" : "B"


    and others




    "_id" : ObjectId("5b08cef10a8a7614c70a5712"),
    "entity" : ObjectId("5b08ceb40a8a7614c70a5710"),
    "name" : "Sub-A"


    "_id" : ObjectId("5b08cefd0a8a7614c70a5713"),
    "entity" : ObjectId("5b08ceb40a8a7614c70a5711"),
    "name" : "Sub-B"



    And a couple of classes to bind them to, just as very basic examples:



    public class Entity

    public ObjectId id;
    public string name get; set;


    public class Other

    public ObjectId id;
    public ObjectId entity get; set;
    public string name get; set;


    public class EntityWithOthers

    public ObjectId id;
    public string name get; set;
    public IEnumerable<Other> others;


    public class EntityWithOther

    public ObjectId id;
    public string name get; set;
    public Other others;



    Queries



    Fluent Interface



    var listNames = new[] "A", "B" ;

    var query = entities.Aggregate()
    .Match(p => listNames.Contains(p.name))
    .Lookup(
    foreignCollection: others,
    localField: e => e.id,
    foreignField: f => f.entity,
    @as: (EntityWithOthers eo) => eo.others
    )
    .Project(p => new p.id, p.name, other = p.others.First() )
    .Sort(new BsonDocument("other.name",-1))
    .ToList();


    Request sent to server:



    [
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "localField" : "_id",
    "foreignField" : "entity",
    "as" : "others"
    ,
    "$project" :
    "id" : "$_id",
    "name" : "$name",
    "other" : "$arrayElemAt" : [ "$others", 0 ] ,
    "_id" : 0
    ,
    "$sort" : "other.name" : -1
    ]


    Probably the easiest to understand since the fluent interface is basically the same as the general BSON structure. The $lookup stage has all the same arguments and the $arrayElemAt is represented with First(). For the $sort you can simply supply a BSON document or other valid expression.



    An alternate is the newer expressive form of $lookup with a sub-pipeline statement for MongoDB 3.6 and above.



    BsonArray subpipeline = new BsonArray();

    subpipeline.Add(
    new BsonDocument("$match",new BsonDocument(
    "$expr", new BsonDocument(
    "$eq", new BsonArray "$$entity", "$entity"
    )
    ))
    );

    var lookup = new BsonDocument("$lookup",
    new BsonDocument("from", "others")
    .Add("let", new BsonDocument("entity", "$_id"))
    .Add("pipeline", subpipeline)
    .Add("as","others")
    );

    var query = entities.Aggregate()
    .Match(p => listNames.Contains(p.name))
    .AppendStage<EntityWithOthers>(lookup)
    .Unwind<EntityWithOthers, EntityWithOther>(p => p.others)
    .SortByDescending(p => p.others.name)
    .ToList();


    Request sent to server:



    [ 
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "let" : "entity" : "$_id" ,
    "pipeline" : [
    "$match" : "$expr" : "$eq" : [ "$$entity", "$entity" ]
    ],
    "as" : "others"
    ,
    "$unwind" : "$others" ,
    "$sort" : "others.name" : -1
    ]


    The Fluent "Builder" does not support the syntax directly yet, nor do LINQ Expressions support the $expr operator, however you can still construct using BsonDocument and BsonArray or other valid expressions. Here we also "type" the $unwind result in order to apply a $sort using an expression rather than a BsonDocument as shown earlier.



    Aside from other uses, a primary task of a "sub-pipeline" is to reduce the documents returned in the target array of $lookup. Also the $unwind here serves a purpose of actually being "merged" into the $lookup statement on server execution, so this is typically more efficient than just grabbing the first element of the resulting array.



    Queryable GroupJoin



    var query = entities.AsQueryable()
    .Where(p => listNames.Contains(p.name))
    .GroupJoin(
    others.AsQueryable(),
    p => p.id,
    o => o.entity,
    (p, o) => new p.id, p.name, other = o.First()
    )
    .OrderByDescending(p => p.other.name);


    Request sent to server:



    [ 
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "localField" : "_id",
    "foreignField" : "entity",
    "as" : "o"
    ,
    "$project" :
    "id" : "$_id",
    "name" : "$name",
    "other" : "$arrayElemAt" : [ "$o", 0 ] ,
    "_id" : 0
    ,
    "$sort" : "other.name" : -1
    ]


    This is almost identical but just using the different interface and produces a slightly different BSON statement, and really only because of the simplified naming in the functional statements. This does bring up the other possibility of simply using an $unwind as produced from a SelectMany():



    var query = entities.AsQueryable()
    .Where(p => listNames.Contains(p.name))
    .GroupJoin(
    others.AsQueryable(),
    p => p.id,
    o => o.entity,
    (p, o) => new p.id, p.name, other = o
    )
    .SelectMany(p => p.other, (p, other) => new p.id, p.name, other )
    .OrderByDescending(p => p.other.name);


    Request sent to server:



    [
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "localField" : "_id",
    "foreignField" : "entity",
    "as" : "o"
    ,
    "$project" :
    "id" : "$_id",
    "name" : "$name",
    "other" : "$o",
    "_id" : 0
    ,
    "$unwind" : "$other" ,
    "$project" :
    "id" : "$id",
    "name" : "$name",
    "other" : "$other",
    "_id" : 0
    ,
    "$sort" : "other.name" : -1
    ]


    Normally placing an $unwind directly following $lookup is actually an "optimized pattern" for the aggregation framework. However the .NET driver does mess this up in this combination by forcing a $project in between rather than using the implied naming on the "as". If not for that, this is actually better than the $arrayElemAt when you know you have "one" related result. If you want the $unwind "coalescence", then you are better off using the fluent interface, or a different form as demonstrated later.



    Querable Natural



    var query = from p in entities.AsQueryable()
    where listNames.Contains(p.name)
    join o in others.AsQueryable() on p.id equals o.entity into joined
    select new p.id, p.name, other = joined.First()
    into p
    orderby p.other.name descending
    select p;


    Request sent to server:



    [
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "localField" : "_id",
    "foreignField" : "entity",
    "as" : "joined"
    ,
    "$project" :
    "id" : "$_id",
    "name" : "$name",
    "other" : "$arrayElemAt" : [ "$joined", 0 ] ,
    "_id" : 0
    ,
    "$sort" : "other.name" : -1
    ]


    All pretty familiar and really just down to functional naming. Just as with using the $unwind option:



    var query = from p in entities.AsQueryable()
    where listNames.Contains(p.name)
    join o in others.AsQueryable() on p.id equals o.entity into joined
    from sub_o in joined.DefaultIfEmpty()
    select new p.id, p.name, other = sub_o
    into p
    orderby p.other.name descending
    select p;


    Request sent to server:



    [ 
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "localField" : "_id",
    "foreignField" : "entity",
    "as" : "joined"
    ,
    "$unwind" :
    "path" : "$joined", "preserveNullAndEmptyArrays" : true
    ,
    "$project" :
    "id" : "$_id",
    "name" : "$name",
    "other" : "$joined",
    "_id" : 0
    ,
    "$sort" : "other.name" : -1
    ]


    Which actually is using the "optimized coalescence" form. The translator still insists on adding a $project since we need the intermediate select in order to make the statement valid.



    Summary



    So there are quite a few ways to essentially arrive at what is basically the same query statement with exactly the same results. Whilst you "could" parse the JSON to BsonDocument form and feed this to the fluent Aggregate() command, it's generally better to use the natural builders or the LINQ interfaces as they do easily map onto the same statement.



    The options with $unwind are largely shown because even with a "singular" match that "coalescence" form is actually far more optimal then using $arrayElemAt to take the "first" array element. This even becomes more important with considerations of things like the BSON Limit where the $lookup target array could cause the parent document to exceed 16MB without further filtering. There is another post here on Aggregate $lookup Total size of documents in matching pipeline exceeds maximum document size where I actually discuss how to avoid that limit being hit by using such options or other Lookup() syntax available to the fluent interface only at this time.






    share|improve this answer






















    • 1





      Really great answer - thanks for taking the time to do this.

      – TomSelleck
      May 28 '18 at 13:01















    28














    There is no need to parse the JSON. Everything here can actually be done directly with either LINQ or the Aggregate Fluent interfaces.



    Just using some demonstration classes because the question does not really give much to go on.



    Setup



    Basically we have two collections here, being



    entities



     "_id" : ObjectId("5b08ceb40a8a7614c70a5710"), "name" : "A" 
    "_id" : ObjectId("5b08ceb40a8a7614c70a5711"), "name" : "B"


    and others




    "_id" : ObjectId("5b08cef10a8a7614c70a5712"),
    "entity" : ObjectId("5b08ceb40a8a7614c70a5710"),
    "name" : "Sub-A"


    "_id" : ObjectId("5b08cefd0a8a7614c70a5713"),
    "entity" : ObjectId("5b08ceb40a8a7614c70a5711"),
    "name" : "Sub-B"



    And a couple of classes to bind them to, just as very basic examples:



    public class Entity

    public ObjectId id;
    public string name get; set;


    public class Other

    public ObjectId id;
    public ObjectId entity get; set;
    public string name get; set;


    public class EntityWithOthers

    public ObjectId id;
    public string name get; set;
    public IEnumerable<Other> others;


    public class EntityWithOther

    public ObjectId id;
    public string name get; set;
    public Other others;



    Queries



    Fluent Interface



    var listNames = new[] "A", "B" ;

    var query = entities.Aggregate()
    .Match(p => listNames.Contains(p.name))
    .Lookup(
    foreignCollection: others,
    localField: e => e.id,
    foreignField: f => f.entity,
    @as: (EntityWithOthers eo) => eo.others
    )
    .Project(p => new p.id, p.name, other = p.others.First() )
    .Sort(new BsonDocument("other.name",-1))
    .ToList();


    Request sent to server:



    [
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "localField" : "_id",
    "foreignField" : "entity",
    "as" : "others"
    ,
    "$project" :
    "id" : "$_id",
    "name" : "$name",
    "other" : "$arrayElemAt" : [ "$others", 0 ] ,
    "_id" : 0
    ,
    "$sort" : "other.name" : -1
    ]


    Probably the easiest to understand since the fluent interface is basically the same as the general BSON structure. The $lookup stage has all the same arguments and the $arrayElemAt is represented with First(). For the $sort you can simply supply a BSON document or other valid expression.



    An alternate is the newer expressive form of $lookup with a sub-pipeline statement for MongoDB 3.6 and above.



    BsonArray subpipeline = new BsonArray();

    subpipeline.Add(
    new BsonDocument("$match",new BsonDocument(
    "$expr", new BsonDocument(
    "$eq", new BsonArray "$$entity", "$entity"
    )
    ))
    );

    var lookup = new BsonDocument("$lookup",
    new BsonDocument("from", "others")
    .Add("let", new BsonDocument("entity", "$_id"))
    .Add("pipeline", subpipeline)
    .Add("as","others")
    );

    var query = entities.Aggregate()
    .Match(p => listNames.Contains(p.name))
    .AppendStage<EntityWithOthers>(lookup)
    .Unwind<EntityWithOthers, EntityWithOther>(p => p.others)
    .SortByDescending(p => p.others.name)
    .ToList();


    Request sent to server:



    [ 
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "let" : "entity" : "$_id" ,
    "pipeline" : [
    "$match" : "$expr" : "$eq" : [ "$$entity", "$entity" ]
    ],
    "as" : "others"
    ,
    "$unwind" : "$others" ,
    "$sort" : "others.name" : -1
    ]


    The Fluent "Builder" does not support the syntax directly yet, nor do LINQ Expressions support the $expr operator, however you can still construct using BsonDocument and BsonArray or other valid expressions. Here we also "type" the $unwind result in order to apply a $sort using an expression rather than a BsonDocument as shown earlier.



    Aside from other uses, a primary task of a "sub-pipeline" is to reduce the documents returned in the target array of $lookup. Also the $unwind here serves a purpose of actually being "merged" into the $lookup statement on server execution, so this is typically more efficient than just grabbing the first element of the resulting array.



    Queryable GroupJoin



    var query = entities.AsQueryable()
    .Where(p => listNames.Contains(p.name))
    .GroupJoin(
    others.AsQueryable(),
    p => p.id,
    o => o.entity,
    (p, o) => new p.id, p.name, other = o.First()
    )
    .OrderByDescending(p => p.other.name);


    Request sent to server:



    [ 
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "localField" : "_id",
    "foreignField" : "entity",
    "as" : "o"
    ,
    "$project" :
    "id" : "$_id",
    "name" : "$name",
    "other" : "$arrayElemAt" : [ "$o", 0 ] ,
    "_id" : 0
    ,
    "$sort" : "other.name" : -1
    ]


    This is almost identical but just using the different interface and produces a slightly different BSON statement, and really only because of the simplified naming in the functional statements. This does bring up the other possibility of simply using an $unwind as produced from a SelectMany():



    var query = entities.AsQueryable()
    .Where(p => listNames.Contains(p.name))
    .GroupJoin(
    others.AsQueryable(),
    p => p.id,
    o => o.entity,
    (p, o) => new p.id, p.name, other = o
    )
    .SelectMany(p => p.other, (p, other) => new p.id, p.name, other )
    .OrderByDescending(p => p.other.name);


    Request sent to server:



    [
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "localField" : "_id",
    "foreignField" : "entity",
    "as" : "o"
    ,
    "$project" :
    "id" : "$_id",
    "name" : "$name",
    "other" : "$o",
    "_id" : 0
    ,
    "$unwind" : "$other" ,
    "$project" :
    "id" : "$id",
    "name" : "$name",
    "other" : "$other",
    "_id" : 0
    ,
    "$sort" : "other.name" : -1
    ]


    Normally placing an $unwind directly following $lookup is actually an "optimized pattern" for the aggregation framework. However the .NET driver does mess this up in this combination by forcing a $project in between rather than using the implied naming on the "as". If not for that, this is actually better than the $arrayElemAt when you know you have "one" related result. If you want the $unwind "coalescence", then you are better off using the fluent interface, or a different form as demonstrated later.



    Querable Natural



    var query = from p in entities.AsQueryable()
    where listNames.Contains(p.name)
    join o in others.AsQueryable() on p.id equals o.entity into joined
    select new p.id, p.name, other = joined.First()
    into p
    orderby p.other.name descending
    select p;


    Request sent to server:



    [
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "localField" : "_id",
    "foreignField" : "entity",
    "as" : "joined"
    ,
    "$project" :
    "id" : "$_id",
    "name" : "$name",
    "other" : "$arrayElemAt" : [ "$joined", 0 ] ,
    "_id" : 0
    ,
    "$sort" : "other.name" : -1
    ]


    All pretty familiar and really just down to functional naming. Just as with using the $unwind option:



    var query = from p in entities.AsQueryable()
    where listNames.Contains(p.name)
    join o in others.AsQueryable() on p.id equals o.entity into joined
    from sub_o in joined.DefaultIfEmpty()
    select new p.id, p.name, other = sub_o
    into p
    orderby p.other.name descending
    select p;


    Request sent to server:



    [ 
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "localField" : "_id",
    "foreignField" : "entity",
    "as" : "joined"
    ,
    "$unwind" :
    "path" : "$joined", "preserveNullAndEmptyArrays" : true
    ,
    "$project" :
    "id" : "$_id",
    "name" : "$name",
    "other" : "$joined",
    "_id" : 0
    ,
    "$sort" : "other.name" : -1
    ]


    Which actually is using the "optimized coalescence" form. The translator still insists on adding a $project since we need the intermediate select in order to make the statement valid.



    Summary



    So there are quite a few ways to essentially arrive at what is basically the same query statement with exactly the same results. Whilst you "could" parse the JSON to BsonDocument form and feed this to the fluent Aggregate() command, it's generally better to use the natural builders or the LINQ interfaces as they do easily map onto the same statement.



    The options with $unwind are largely shown because even with a "singular" match that "coalescence" form is actually far more optimal then using $arrayElemAt to take the "first" array element. This even becomes more important with considerations of things like the BSON Limit where the $lookup target array could cause the parent document to exceed 16MB without further filtering. There is another post here on Aggregate $lookup Total size of documents in matching pipeline exceeds maximum document size where I actually discuss how to avoid that limit being hit by using such options or other Lookup() syntax available to the fluent interface only at this time.






    share|improve this answer






















    • 1





      Really great answer - thanks for taking the time to do this.

      – TomSelleck
      May 28 '18 at 13:01













    28












    28








    28







    There is no need to parse the JSON. Everything here can actually be done directly with either LINQ or the Aggregate Fluent interfaces.



    Just using some demonstration classes because the question does not really give much to go on.



    Setup



    Basically we have two collections here, being



    entities



     "_id" : ObjectId("5b08ceb40a8a7614c70a5710"), "name" : "A" 
    "_id" : ObjectId("5b08ceb40a8a7614c70a5711"), "name" : "B"


    and others




    "_id" : ObjectId("5b08cef10a8a7614c70a5712"),
    "entity" : ObjectId("5b08ceb40a8a7614c70a5710"),
    "name" : "Sub-A"


    "_id" : ObjectId("5b08cefd0a8a7614c70a5713"),
    "entity" : ObjectId("5b08ceb40a8a7614c70a5711"),
    "name" : "Sub-B"



    And a couple of classes to bind them to, just as very basic examples:



    public class Entity

    public ObjectId id;
    public string name get; set;


    public class Other

    public ObjectId id;
    public ObjectId entity get; set;
    public string name get; set;


    public class EntityWithOthers

    public ObjectId id;
    public string name get; set;
    public IEnumerable<Other> others;


    public class EntityWithOther

    public ObjectId id;
    public string name get; set;
    public Other others;



    Queries



    Fluent Interface



    var listNames = new[] "A", "B" ;

    var query = entities.Aggregate()
    .Match(p => listNames.Contains(p.name))
    .Lookup(
    foreignCollection: others,
    localField: e => e.id,
    foreignField: f => f.entity,
    @as: (EntityWithOthers eo) => eo.others
    )
    .Project(p => new p.id, p.name, other = p.others.First() )
    .Sort(new BsonDocument("other.name",-1))
    .ToList();


    Request sent to server:



    [
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "localField" : "_id",
    "foreignField" : "entity",
    "as" : "others"
    ,
    "$project" :
    "id" : "$_id",
    "name" : "$name",
    "other" : "$arrayElemAt" : [ "$others", 0 ] ,
    "_id" : 0
    ,
    "$sort" : "other.name" : -1
    ]


    Probably the easiest to understand since the fluent interface is basically the same as the general BSON structure. The $lookup stage has all the same arguments and the $arrayElemAt is represented with First(). For the $sort you can simply supply a BSON document or other valid expression.



    An alternate is the newer expressive form of $lookup with a sub-pipeline statement for MongoDB 3.6 and above.



    BsonArray subpipeline = new BsonArray();

    subpipeline.Add(
    new BsonDocument("$match",new BsonDocument(
    "$expr", new BsonDocument(
    "$eq", new BsonArray "$$entity", "$entity"
    )
    ))
    );

    var lookup = new BsonDocument("$lookup",
    new BsonDocument("from", "others")
    .Add("let", new BsonDocument("entity", "$_id"))
    .Add("pipeline", subpipeline)
    .Add("as","others")
    );

    var query = entities.Aggregate()
    .Match(p => listNames.Contains(p.name))
    .AppendStage<EntityWithOthers>(lookup)
    .Unwind<EntityWithOthers, EntityWithOther>(p => p.others)
    .SortByDescending(p => p.others.name)
    .ToList();


    Request sent to server:



    [ 
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "let" : "entity" : "$_id" ,
    "pipeline" : [
    "$match" : "$expr" : "$eq" : [ "$$entity", "$entity" ]
    ],
    "as" : "others"
    ,
    "$unwind" : "$others" ,
    "$sort" : "others.name" : -1
    ]


    The Fluent "Builder" does not support the syntax directly yet, nor do LINQ Expressions support the $expr operator, however you can still construct using BsonDocument and BsonArray or other valid expressions. Here we also "type" the $unwind result in order to apply a $sort using an expression rather than a BsonDocument as shown earlier.



    Aside from other uses, a primary task of a "sub-pipeline" is to reduce the documents returned in the target array of $lookup. Also the $unwind here serves a purpose of actually being "merged" into the $lookup statement on server execution, so this is typically more efficient than just grabbing the first element of the resulting array.



    Queryable GroupJoin



    var query = entities.AsQueryable()
    .Where(p => listNames.Contains(p.name))
    .GroupJoin(
    others.AsQueryable(),
    p => p.id,
    o => o.entity,
    (p, o) => new p.id, p.name, other = o.First()
    )
    .OrderByDescending(p => p.other.name);


    Request sent to server:



    [ 
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "localField" : "_id",
    "foreignField" : "entity",
    "as" : "o"
    ,
    "$project" :
    "id" : "$_id",
    "name" : "$name",
    "other" : "$arrayElemAt" : [ "$o", 0 ] ,
    "_id" : 0
    ,
    "$sort" : "other.name" : -1
    ]


    This is almost identical but just using the different interface and produces a slightly different BSON statement, and really only because of the simplified naming in the functional statements. This does bring up the other possibility of simply using an $unwind as produced from a SelectMany():



    var query = entities.AsQueryable()
    .Where(p => listNames.Contains(p.name))
    .GroupJoin(
    others.AsQueryable(),
    p => p.id,
    o => o.entity,
    (p, o) => new p.id, p.name, other = o
    )
    .SelectMany(p => p.other, (p, other) => new p.id, p.name, other )
    .OrderByDescending(p => p.other.name);


    Request sent to server:



    [
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "localField" : "_id",
    "foreignField" : "entity",
    "as" : "o"
    ,
    "$project" :
    "id" : "$_id",
    "name" : "$name",
    "other" : "$o",
    "_id" : 0
    ,
    "$unwind" : "$other" ,
    "$project" :
    "id" : "$id",
    "name" : "$name",
    "other" : "$other",
    "_id" : 0
    ,
    "$sort" : "other.name" : -1
    ]


    Normally placing an $unwind directly following $lookup is actually an "optimized pattern" for the aggregation framework. However the .NET driver does mess this up in this combination by forcing a $project in between rather than using the implied naming on the "as". If not for that, this is actually better than the $arrayElemAt when you know you have "one" related result. If you want the $unwind "coalescence", then you are better off using the fluent interface, or a different form as demonstrated later.



    Querable Natural



    var query = from p in entities.AsQueryable()
    where listNames.Contains(p.name)
    join o in others.AsQueryable() on p.id equals o.entity into joined
    select new p.id, p.name, other = joined.First()
    into p
    orderby p.other.name descending
    select p;


    Request sent to server:



    [
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "localField" : "_id",
    "foreignField" : "entity",
    "as" : "joined"
    ,
    "$project" :
    "id" : "$_id",
    "name" : "$name",
    "other" : "$arrayElemAt" : [ "$joined", 0 ] ,
    "_id" : 0
    ,
    "$sort" : "other.name" : -1
    ]


    All pretty familiar and really just down to functional naming. Just as with using the $unwind option:



    var query = from p in entities.AsQueryable()
    where listNames.Contains(p.name)
    join o in others.AsQueryable() on p.id equals o.entity into joined
    from sub_o in joined.DefaultIfEmpty()
    select new p.id, p.name, other = sub_o
    into p
    orderby p.other.name descending
    select p;


    Request sent to server:



    [ 
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "localField" : "_id",
    "foreignField" : "entity",
    "as" : "joined"
    ,
    "$unwind" :
    "path" : "$joined", "preserveNullAndEmptyArrays" : true
    ,
    "$project" :
    "id" : "$_id",
    "name" : "$name",
    "other" : "$joined",
    "_id" : 0
    ,
    "$sort" : "other.name" : -1
    ]


    Which actually is using the "optimized coalescence" form. The translator still insists on adding a $project since we need the intermediate select in order to make the statement valid.



    Summary



    So there are quite a few ways to essentially arrive at what is basically the same query statement with exactly the same results. Whilst you "could" parse the JSON to BsonDocument form and feed this to the fluent Aggregate() command, it's generally better to use the natural builders or the LINQ interfaces as they do easily map onto the same statement.



    The options with $unwind are largely shown because even with a "singular" match that "coalescence" form is actually far more optimal then using $arrayElemAt to take the "first" array element. This even becomes more important with considerations of things like the BSON Limit where the $lookup target array could cause the parent document to exceed 16MB without further filtering. There is another post here on Aggregate $lookup Total size of documents in matching pipeline exceeds maximum document size where I actually discuss how to avoid that limit being hit by using such options or other Lookup() syntax available to the fluent interface only at this time.






    share|improve this answer















    There is no need to parse the JSON. Everything here can actually be done directly with either LINQ or the Aggregate Fluent interfaces.



    Just using some demonstration classes because the question does not really give much to go on.



    Setup



    Basically we have two collections here, being



    entities



     "_id" : ObjectId("5b08ceb40a8a7614c70a5710"), "name" : "A" 
    "_id" : ObjectId("5b08ceb40a8a7614c70a5711"), "name" : "B"


    and others




    "_id" : ObjectId("5b08cef10a8a7614c70a5712"),
    "entity" : ObjectId("5b08ceb40a8a7614c70a5710"),
    "name" : "Sub-A"


    "_id" : ObjectId("5b08cefd0a8a7614c70a5713"),
    "entity" : ObjectId("5b08ceb40a8a7614c70a5711"),
    "name" : "Sub-B"



    And a couple of classes to bind them to, just as very basic examples:



    public class Entity

    public ObjectId id;
    public string name get; set;


    public class Other

    public ObjectId id;
    public ObjectId entity get; set;
    public string name get; set;


    public class EntityWithOthers

    public ObjectId id;
    public string name get; set;
    public IEnumerable<Other> others;


    public class EntityWithOther

    public ObjectId id;
    public string name get; set;
    public Other others;



    Queries



    Fluent Interface



    var listNames = new[] "A", "B" ;

    var query = entities.Aggregate()
    .Match(p => listNames.Contains(p.name))
    .Lookup(
    foreignCollection: others,
    localField: e => e.id,
    foreignField: f => f.entity,
    @as: (EntityWithOthers eo) => eo.others
    )
    .Project(p => new p.id, p.name, other = p.others.First() )
    .Sort(new BsonDocument("other.name",-1))
    .ToList();


    Request sent to server:



    [
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "localField" : "_id",
    "foreignField" : "entity",
    "as" : "others"
    ,
    "$project" :
    "id" : "$_id",
    "name" : "$name",
    "other" : "$arrayElemAt" : [ "$others", 0 ] ,
    "_id" : 0
    ,
    "$sort" : "other.name" : -1
    ]


    Probably the easiest to understand since the fluent interface is basically the same as the general BSON structure. The $lookup stage has all the same arguments and the $arrayElemAt is represented with First(). For the $sort you can simply supply a BSON document or other valid expression.



    An alternate is the newer expressive form of $lookup with a sub-pipeline statement for MongoDB 3.6 and above.



    BsonArray subpipeline = new BsonArray();

    subpipeline.Add(
    new BsonDocument("$match",new BsonDocument(
    "$expr", new BsonDocument(
    "$eq", new BsonArray "$$entity", "$entity"
    )
    ))
    );

    var lookup = new BsonDocument("$lookup",
    new BsonDocument("from", "others")
    .Add("let", new BsonDocument("entity", "$_id"))
    .Add("pipeline", subpipeline)
    .Add("as","others")
    );

    var query = entities.Aggregate()
    .Match(p => listNames.Contains(p.name))
    .AppendStage<EntityWithOthers>(lookup)
    .Unwind<EntityWithOthers, EntityWithOther>(p => p.others)
    .SortByDescending(p => p.others.name)
    .ToList();


    Request sent to server:



    [ 
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "let" : "entity" : "$_id" ,
    "pipeline" : [
    "$match" : "$expr" : "$eq" : [ "$$entity", "$entity" ]
    ],
    "as" : "others"
    ,
    "$unwind" : "$others" ,
    "$sort" : "others.name" : -1
    ]


    The Fluent "Builder" does not support the syntax directly yet, nor do LINQ Expressions support the $expr operator, however you can still construct using BsonDocument and BsonArray or other valid expressions. Here we also "type" the $unwind result in order to apply a $sort using an expression rather than a BsonDocument as shown earlier.



    Aside from other uses, a primary task of a "sub-pipeline" is to reduce the documents returned in the target array of $lookup. Also the $unwind here serves a purpose of actually being "merged" into the $lookup statement on server execution, so this is typically more efficient than just grabbing the first element of the resulting array.



    Queryable GroupJoin



    var query = entities.AsQueryable()
    .Where(p => listNames.Contains(p.name))
    .GroupJoin(
    others.AsQueryable(),
    p => p.id,
    o => o.entity,
    (p, o) => new p.id, p.name, other = o.First()
    )
    .OrderByDescending(p => p.other.name);


    Request sent to server:



    [ 
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "localField" : "_id",
    "foreignField" : "entity",
    "as" : "o"
    ,
    "$project" :
    "id" : "$_id",
    "name" : "$name",
    "other" : "$arrayElemAt" : [ "$o", 0 ] ,
    "_id" : 0
    ,
    "$sort" : "other.name" : -1
    ]


    This is almost identical but just using the different interface and produces a slightly different BSON statement, and really only because of the simplified naming in the functional statements. This does bring up the other possibility of simply using an $unwind as produced from a SelectMany():



    var query = entities.AsQueryable()
    .Where(p => listNames.Contains(p.name))
    .GroupJoin(
    others.AsQueryable(),
    p => p.id,
    o => o.entity,
    (p, o) => new p.id, p.name, other = o
    )
    .SelectMany(p => p.other, (p, other) => new p.id, p.name, other )
    .OrderByDescending(p => p.other.name);


    Request sent to server:



    [
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "localField" : "_id",
    "foreignField" : "entity",
    "as" : "o"
    ,
    "$project" :
    "id" : "$_id",
    "name" : "$name",
    "other" : "$o",
    "_id" : 0
    ,
    "$unwind" : "$other" ,
    "$project" :
    "id" : "$id",
    "name" : "$name",
    "other" : "$other",
    "_id" : 0
    ,
    "$sort" : "other.name" : -1
    ]


    Normally placing an $unwind directly following $lookup is actually an "optimized pattern" for the aggregation framework. However the .NET driver does mess this up in this combination by forcing a $project in between rather than using the implied naming on the "as". If not for that, this is actually better than the $arrayElemAt when you know you have "one" related result. If you want the $unwind "coalescence", then you are better off using the fluent interface, or a different form as demonstrated later.



    Querable Natural



    var query = from p in entities.AsQueryable()
    where listNames.Contains(p.name)
    join o in others.AsQueryable() on p.id equals o.entity into joined
    select new p.id, p.name, other = joined.First()
    into p
    orderby p.other.name descending
    select p;


    Request sent to server:



    [
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "localField" : "_id",
    "foreignField" : "entity",
    "as" : "joined"
    ,
    "$project" :
    "id" : "$_id",
    "name" : "$name",
    "other" : "$arrayElemAt" : [ "$joined", 0 ] ,
    "_id" : 0
    ,
    "$sort" : "other.name" : -1
    ]


    All pretty familiar and really just down to functional naming. Just as with using the $unwind option:



    var query = from p in entities.AsQueryable()
    where listNames.Contains(p.name)
    join o in others.AsQueryable() on p.id equals o.entity into joined
    from sub_o in joined.DefaultIfEmpty()
    select new p.id, p.name, other = sub_o
    into p
    orderby p.other.name descending
    select p;


    Request sent to server:



    [ 
    "$match" : "name" : "$in" : [ "A", "B" ] ,
    "$lookup" :
    "from" : "others",
    "localField" : "_id",
    "foreignField" : "entity",
    "as" : "joined"
    ,
    "$unwind" :
    "path" : "$joined", "preserveNullAndEmptyArrays" : true
    ,
    "$project" :
    "id" : "$_id",
    "name" : "$name",
    "other" : "$joined",
    "_id" : 0
    ,
    "$sort" : "other.name" : -1
    ]


    Which actually is using the "optimized coalescence" form. The translator still insists on adding a $project since we need the intermediate select in order to make the statement valid.



    Summary



    So there are quite a few ways to essentially arrive at what is basically the same query statement with exactly the same results. Whilst you "could" parse the JSON to BsonDocument form and feed this to the fluent Aggregate() command, it's generally better to use the natural builders or the LINQ interfaces as they do easily map onto the same statement.



    The options with $unwind are largely shown because even with a "singular" match that "coalescence" form is actually far more optimal then using $arrayElemAt to take the "first" array element. This even becomes more important with considerations of things like the BSON Limit where the $lookup target array could cause the parent document to exceed 16MB without further filtering. There is another post here on Aggregate $lookup Total size of documents in matching pipeline exceeds maximum document size where I actually discuss how to avoid that limit being hit by using such options or other Lookup() syntax available to the fluent interface only at this time.







    share|improve this answer














    share|improve this answer



    share|improve this answer








    edited May 27 '18 at 0:41

























    answered May 26 '18 at 6:38









    Neil LunnNeil Lunn

    106k24 gold badges195 silver badges199 bronze badges




    106k24 gold badges195 silver badges199 bronze badges










    • 1





      Really great answer - thanks for taking the time to do this.

      – TomSelleck
      May 28 '18 at 13:01












    • 1





      Really great answer - thanks for taking the time to do this.

      – TomSelleck
      May 28 '18 at 13:01







    1




    1





    Really great answer - thanks for taking the time to do this.

    – TomSelleck
    May 28 '18 at 13:01





    Really great answer - thanks for taking the time to do this.

    – TomSelleck
    May 28 '18 at 13:01













    0














    here's how to do it with MongoDB.Entities. in cases where two entities are in either a one-to-many or many-to-many relationship, you can get reverse relationship access without having to do joins manually as shown below. [disclaimer: i'm the author of the library]



    using System;
    using System.Linq;
    using MongoDB.Entities;
    using MongoDB.Driver.Linq;

    namespace StackOverflow

    public class Program

    public class Author : Entity

    public string Name get; set;
    public Many<Book> Books get; set;

    public Author() => this.InitOneToMany(() => Books);


    public class Book : Entity

    public string Title get; set;


    static void Main(string[] args)

    new DB("test");

    var book = new Book Title = "The Power Of Now" ;
    book.Save();

    var author = new Author Name = "Eckhart Tolle" ;
    author.Save();

    author.Books.Add(book);

    //build a query for finding all books that has Power in the title.
    var bookQuery = DB.Queryable<Book>()
    .Where(b => b.Title.Contains("Power"));

    //find all the authors of books that has a title with Power in them
    var authors = author.Books
    .ParentsQueryable<Author>(bookQuery); //also can pass in an ID or array of IDs

    //get the result
    var result = authors.ToArray();

    //output the aggregation pipeline
    Console.WriteLine(authors.ToString());


    Console.ReadKey();








    share|improve this answer





























      0














      here's how to do it with MongoDB.Entities. in cases where two entities are in either a one-to-many or many-to-many relationship, you can get reverse relationship access without having to do joins manually as shown below. [disclaimer: i'm the author of the library]



      using System;
      using System.Linq;
      using MongoDB.Entities;
      using MongoDB.Driver.Linq;

      namespace StackOverflow

      public class Program

      public class Author : Entity

      public string Name get; set;
      public Many<Book> Books get; set;

      public Author() => this.InitOneToMany(() => Books);


      public class Book : Entity

      public string Title get; set;


      static void Main(string[] args)

      new DB("test");

      var book = new Book Title = "The Power Of Now" ;
      book.Save();

      var author = new Author Name = "Eckhart Tolle" ;
      author.Save();

      author.Books.Add(book);

      //build a query for finding all books that has Power in the title.
      var bookQuery = DB.Queryable<Book>()
      .Where(b => b.Title.Contains("Power"));

      //find all the authors of books that has a title with Power in them
      var authors = author.Books
      .ParentsQueryable<Author>(bookQuery); //also can pass in an ID or array of IDs

      //get the result
      var result = authors.ToArray();

      //output the aggregation pipeline
      Console.WriteLine(authors.ToString());


      Console.ReadKey();








      share|improve this answer



























        0












        0








        0







        here's how to do it with MongoDB.Entities. in cases where two entities are in either a one-to-many or many-to-many relationship, you can get reverse relationship access without having to do joins manually as shown below. [disclaimer: i'm the author of the library]



        using System;
        using System.Linq;
        using MongoDB.Entities;
        using MongoDB.Driver.Linq;

        namespace StackOverflow

        public class Program

        public class Author : Entity

        public string Name get; set;
        public Many<Book> Books get; set;

        public Author() => this.InitOneToMany(() => Books);


        public class Book : Entity

        public string Title get; set;


        static void Main(string[] args)

        new DB("test");

        var book = new Book Title = "The Power Of Now" ;
        book.Save();

        var author = new Author Name = "Eckhart Tolle" ;
        author.Save();

        author.Books.Add(book);

        //build a query for finding all books that has Power in the title.
        var bookQuery = DB.Queryable<Book>()
        .Where(b => b.Title.Contains("Power"));

        //find all the authors of books that has a title with Power in them
        var authors = author.Books
        .ParentsQueryable<Author>(bookQuery); //also can pass in an ID or array of IDs

        //get the result
        var result = authors.ToArray();

        //output the aggregation pipeline
        Console.WriteLine(authors.ToString());


        Console.ReadKey();








        share|improve this answer













        here's how to do it with MongoDB.Entities. in cases where two entities are in either a one-to-many or many-to-many relationship, you can get reverse relationship access without having to do joins manually as shown below. [disclaimer: i'm the author of the library]



        using System;
        using System.Linq;
        using MongoDB.Entities;
        using MongoDB.Driver.Linq;

        namespace StackOverflow

        public class Program

        public class Author : Entity

        public string Name get; set;
        public Many<Book> Books get; set;

        public Author() => this.InitOneToMany(() => Books);


        public class Book : Entity

        public string Title get; set;


        static void Main(string[] args)

        new DB("test");

        var book = new Book Title = "The Power Of Now" ;
        book.Save();

        var author = new Author Name = "Eckhart Tolle" ;
        author.Save();

        author.Books.Add(book);

        //build a query for finding all books that has Power in the title.
        var bookQuery = DB.Queryable<Book>()
        .Where(b => b.Title.Contains("Power"));

        //find all the authors of books that has a title with Power in them
        var authors = author.Books
        .ParentsQueryable<Author>(bookQuery); //also can pass in an ID or array of IDs

        //get the result
        var result = authors.ToArray();

        //output the aggregation pipeline
        Console.WriteLine(authors.ToString());


        Console.ReadKey();









        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Jun 4 at 13:12









        Ryan GunnerRyan Gunner

        8211 gold badge6 silver badges12 bronze badges




        8211 gold badge6 silver badges12 bronze badges






























            draft saved

            draft discarded
















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f50530363%2faggregate-lookup-with-c-sharp%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Kamusi Yaliyomo Aina za kamusi | Muundo wa kamusi | Faida za kamusi | Dhima ya picha katika kamusi | Marejeo | Tazama pia | Viungo vya nje | UrambazajiKuhusu kamusiGo-SwahiliWiki-KamusiKamusi ya Kiswahili na Kiingerezakuihariri na kuongeza habari

            Swift 4 - func physicsWorld not invoked on collision? The Next CEO of Stack OverflowHow to call Objective-C code from Swift#ifdef replacement in the Swift language@selector() in Swift?#pragma mark in Swift?Swift for loop: for index, element in array?dispatch_after - GCD in Swift?Swift Beta performance: sorting arraysSplit a String into an array in Swift?The use of Swift 3 @objc inference in Swift 4 mode is deprecated?How to optimize UITableViewCell, because my UITableView lags

            Access current req object everywhere in Node.js ExpressWhy are global variables considered bad practice? (node.js)Using req & res across functionsHow do I get the path to the current script with Node.js?What is Node.js' Connect, Express and “middleware”?Node.js w/ express error handling in callbackHow to access the GET parameters after “?” in Express?Modify Node.js req object parametersAccess “app” variable inside of ExpressJS/ConnectJS middleware?Node.js Express app - request objectAngular Http Module considered middleware?Session variables in ExpressJSAdd properties to the req object in expressjs with Typescript