PRO 30% Free • Stripe ($15) & Razorpay (₹899)

MongoDB Aggregation Stages Explained

Top Course Staff · 8 min read · Web development · PRO · 30% Free Preview Available

MongoDB Aggregation Stages Explained

While analyzing data, we often need to filter, sort, and aggregate it to calculate metrics like sum, count, and average for different values in the dataset. For example, let’s say we want information about the average salary of employees in a particular department of an organization.

To solve this problem, we need to filter data, calculate the average salaries, and sort the data as required. We can perform these operations using different stages in MongoDB aggregation pipelines.

To fully grasp the material covered in this tutorial, you should be familiar with the basics of MongoDB. For this, you canlearn the basics of MongoDBwith our free course.

Let’s discuss MongoDB aggregation stages such as$match,$group,$project, etc. and how we can use them to extract insights from data.

Dataset Overview and Preparation

To understand how the different stages in MongoDB aggregation pipelines work, we will use a sample employee dataset that contains the following fields:

  • employee ID
  • employee name
  • department ID
  • salary
  • employee age

To prepare the dataset, we start the MongoDB shell interface using themongoshcommand in a command-line application. We will get atestprompt like the one shown below:

test>  
Copy to clipboard
test>

For our experiments, we will create a new database namedcodecademy. For this, we can execute the commanduse codecademy.

test> use codecademy;  switched to db codecademy  codecademy>   
Copy to clipboard
test> use codecademy;  switched to db codecademy  codecademy>

After creating the database, we create a collection namedemployeesthat will store the data for the employees using thedb.createCollection()function. ThecreateCollection()function takes the collection name as its input argument and creates a new collection in the current dataset.

codecademy> db.createCollection("employees");  
Copy to clipboard
codecademy> db.createCollection("employees");

Next, the data will be inserted into the collection using theinsertMany()method, as shown below:

codecademy> db.employees.insertMany([{ "emp_id": 1, "emp_name": "Ankit", "dept_id": 100, "salary": 10000, "emp_age": 32 }, { "emp_id": 2, "emp_name": "Aditya", "dept_id": 100, "salary": 12000, "emp_age": 25 }, { "emp_id": 3, "emp_name": "Adam", "dept_id": 100, "salary": 13000, "emp_age": 27 }, { "emp_id": 4, "emp_name": "Katy", "dept_id": 100, "salary": 12000, "emp_age": 23 }, { "emp_id": 5, "emp_name": "Akshat", "dept_id": 200, "salary": 15000, "emp_age": 21 }, { "emp_id": 6, "emp_name": "Jasmine", "dept_id": 200, "salary": 14000, "emp_age": 24 }, { "emp_id": 7, "emp_name": "Samantha", "dept_id": 200, "salary": 16000, "emp_age": 27 }, { "emp_id": 8, "emp_name": "Cole", "dept_id": 300, "salary": 20000, "emp_age": 36 }, { "emp_id": 9, "emp_name": "Kunal", "dept_id": 300, "salary": 25000, "emp_age": 39 }, { "emp_id": 10, "emp_name": "Joe", "dept_id": 400, "salary": 15000, "emp_age": 25 }])
Copy to clipboard
codecademy> db.employees.insertMany([{ "emp_id": 1, "emp_name": "Ankit", "dept_id": 100, "salary": 10000, "emp_age": 32 }, { "emp_id": 2, "emp_name": "Aditya", "dept_id": 100, "salary": 12000, "emp_age": 25 }, { "emp_id": 3, "emp_name": "Adam", "dept_id": 100, "salary": 13000, "emp_age": 27 }, { "emp_id": 4, "emp_name": "Katy", "dept_id": 100, "salary": 12000, "emp_age": 23 }, { "emp_id": 5, "emp_name": "Akshat", "dept_id": 200, "salary": 15000, "emp_age": 21 }, { "emp_id": 6, "emp_name": "Jasmine", "dept_id": 200, "salary": 14000, "emp_age": 24 }, { "emp_id": 7, "emp_name": "Samantha", "dept_id": 200, "salary": 16000, "emp_age": 27 }, { "emp_id": 8, "emp_name": "Cole", "dept_id": 300, "salary": 20000, "emp_age": 36 }, { "emp_id": 9, "emp_name": "Kunal", "dept_id": 300, "salary": 25000, "emp_age": 39 }, { "emp_id": 10, "emp_name": "Joe", "dept_id": 400, "salary": 15000, "emp_age": 25 }])

After executing theinsertMany()function, we get an output showing that the documents are inserted successfully.

Now that the dataset is ready, let’s understand how each stage in MongoDB aggregation pipelines works.

How do MongoDB Aggregation Stages Work?

MongoDB aggregation pipelines consist of one or morestagesthat perform specific operations on documents.

A particular stage in the MongoDB pipeline filters data, groups documents, sorts them, or selects specific fields from the documents. The most common stages of MongoDB aggregation pipelines are$match,$group,$sort, and$project. To combine different stages together and execute the MongoDB pipelines, we use theaggregate()function.

The aggregate() Function

Theaggregate()function allows us to utilize various aggregation stages and operators to derive insights from the data. It has the following syntax:

db.collection_name.aggregate(  
[  
  stage 1,  
  stage 2,   
  stage 3,  
  …….  
  stage N  
]  
)

In the above syntax,

  • The keyworddbrefers to the current database.
  • collection_nameis the MongoDB collection in which the data is stored.
  • Stages 1, 2, 3, and N are stages like$match,$group,$sort, etc.
  • Every stage, after its execution, returns a document.
  • Stage 1 reads data from thecollection_namecollection as input. The output document from each stage is used as input for the next stage. The final stage’s output is considered the output of the entire aggregation pipeline.

In theaggregate()function, each stage can use zero or more aggregation operators, such as$sum,$count, etc., as required. Except for afew particular stages, a specific stage can be used multiple times in an aggregation pipeline.

How do the different stages and theaggregate()function work? To understand this, we will discuss the different stages individually.

Different Stages in MongoDB Aggregation Pipelines

MongoDB aggregation pipelines consist of various stages, such as$match,$group,$sort, and$project.

The$matchStage

The$matchstage is used to filter data from a MongoDB collection. To use the$matchstage, we use the following syntax.

db.collection_name.aggregate(  

[{  
  $match:{  
    field_name1:value,  
    field_name2:value  
  }  
}])

In the above syntax,

  • The document passed to the$matchstage can contain zero or more field names and their desired values.
  • When we specify the field names and their values in the document passed to the$matchstage, only the documents with the same values as the input fields are returned as output. If no field names exist in the document, we get all the documents in thecollection_namecollection as the output.

To understand this stage, let’s find all the documents in theemployeescollection with thedept_idvalue as300. As shown below, we will pass the document{"dept_id":300}to the$matchstage:

codecademy> db.employees.aggregate([{$match:{"dept_id":300}}])  
Copy to clipboard
codecademy> db.employees.aggregate([{$match:{"dept_id":300}}])

After executing the above code, we get the following output:

[    {      _id: ObjectId('6636483d3e7fbdb6982202df'),      emp_id: 8,      emp_name: 'Cole',      dept_id: 300,      salary: 20000,      emp_age: 36    },    {      _id: ObjectId('6636483d3e7fbdb6982202e0'),      emp_id: 9,      emp_name: 'Kunal',      dept_id: 300,      salary: 25000,      emp_age: 39    }  ]  
Copy to clipboard
[    {      _id: ObjectId('6636483d3e7fbdb6982202df'),      emp_id: 8,      emp_name: 'Cole',      dept_id: 300,      salary: 20000,      emp_age: 36    },    {      _id: ObjectId('6636483d3e7fbdb6982202e0'),      emp_id: 9,      emp_name: 'Kunal',      dept_id: 300,      salary: 25000,      emp_age: 39    }  ]

We can also add multiple field names in the document passed to the$matchstage. For instance, to get all the documents with thedept_id100 andsalary12000, we will pass the document{"dept_id":100, "salary":12000}to the$matchstage:

codecademy> db.employees.aggregate([{$match:{"dept_id":100,"salary":12000}}])  
Copy to clipboard
codecademy> db.employees.aggregate([{$match:{"dept_id":100,"salary":12000}}])

The output for the above code is as follows:

[    {      _id: ObjectId('6636483d3e7fbdb6982202d9'),      emp_id: 2,      emp_name: 'Aditya',      dept_id: 100,      salary: 12000,      emp_age: 25    },    {      _id: ObjectId('6636483d3e7fbdb6982202db'),      emp_id: 4,      emp_name: 'Katy',      dept_id: 100,      salary: 12000,      emp_age: 23    }  ]  
Copy to clipboard
[    {      _id: ObjectId('6636483d3e7fbdb6982202d9'),      emp_id: 2,      emp_name: 'Aditya',      dept_id: 100,      salary: 12000,      emp_age: 25    },    {      _id: ObjectId('6636483d3e7fbdb6982202db'),      emp_id: 4,      emp_name: 'Katy',      dept_id: 100,      salary: 12000,      emp_age: 23    }  ]

Apart from filtering data, we might also need to group data using values in specific fields such as age, department, and others. We use the$groupstage in the MongoDB aggregation pipeline for this.

The$groupStage

We use the$groupstage to group documents based on specific data fields and calculate aggregate values such as sum, average, min, or max for one or more fields.

The syntax for the$groupstage is as follows:

db.collection_name.aggregate(  
[  
{  
 $group:{  
  _id:{  
    field_name1:"$field_name1",  
    field_name2:"$field_name2"	  
  },  
    new_value={ $operator:"$fieldname"}  
  }  
}  
])

In the above syntax,

  • We pass a document containing any number of field names to the_idkey. After execution, the$groupstage returns all the documents in the given collection grouped by the values in the provided fields.
  • When multiple field names are present in the document passed to the_idkey, each unique set of values in the given field names is considered a separate group.
  • When we pass the valuenullto the_idkey, the$groupstage outputs all the documents in a single group.
  • new_valueis calculated for each group.
  • The$operatoris an aggregation operator like$sum,$count,$avg, etc., andfieldnameis a field name from documents. Using the aggregation operators with the$groupstage is optional.

To understand how the$groupstage works, let’s use the given data to count the number of employees in each department.

For this, we will group the documents in theemployeescollection by thedept_idfield by passing the document{ dept_id:"$dept_id" }to the_idkey in the$groupstage.

To count the number of employees in each department, we will use the$countoperator. The$countoperator takes an empty document{}as its input.

codecademy> db.employees.aggregate([{$group: {_id: {dept_id:"$dept_id"}, number_of_employees:{$count:{}}}}])  
Copy to clipboard
codecademy> db.employees.aggregate([{$group: {_id: {dept_id:"$dept_id"}, number_of_employees:{$count:{}}}}])

The output from the above command is the following:

[    { _id: { dept_id: 100 }, number_of_employees: 4 },    { _id: { dept_id: 200 }, number_of_employees: 3 },    { _id: { dept_id: 400 }, number_of_employees: 1 },    { _id: { dept_id: 300 }, number_of_employees: 2 }  ]  
Copy to clipboard
[    { _id: { dept_id: 100 }, number_of_employees: 4 },    { _id: { dept_id: 200 }, number_of_employees: 3 },    { _id: { dept_id: 400 }, number_of_employees: 1 },    { _id: { dept_id: 300 }, number_of_employees: 2 }  ]

In the output, you can see that we got four documents, each containing the department ID and the number of employees in that department.

We can also use multiple fields to group the data. For example, to count the number of employees that get a particular salary in each department, we can pass the document{ dept_id:"$dept_id", salary:"$salary" }to theid_key in the$groupstage:

codecademy> db.employees.aggregate([{$group: {_id: {dept_id:"$dept_id",salary:"$salary"}, number_of_employees:{$count:{}}}}])  
Copy to clipboard
codecademy> db.employees.aggregate([{$group: {_id: {dept_id:"$dept_id",salary:"$salary"}, number_of_employees:{$count:{}}}}])

Output:

[    { _id: { dept_id: 200, salary: 15000 }, number_of_employees: 1 },    { _id: { dept_id: 200, salary: 16000 }, number_of_employees: 1 },    { _id: { dept_id: 100, salary: 13000 }, number_of_employees: 1 },    { _id: { dept_id: 200, salary: 14000 }, number_of_employees: 1 },    { _id: { dept_id: 300, salary: 25000 }, number_of_employees: 1 },    { _id: { dept_id: 400, salary: 15000 }, number_of_employees: 1 },    { _id: { dept_id: 300, salary: 20000 }, number_of_employees: 1 },    { _id: { dept_id: 100, salary: 10000 }, number_of_employees: 1 },    { _id: { dept_id: 100, salary: 12000 }, number_of_employees: 2 }  ]  
Copy to clipboard
[    { _id: { dept_id: 200, salary: 15000 }, number_of_employees: 1 },    { _id: { dept_id: 200, salary: 16000 }, number_of_employees: 1 },    { _id: { dept_id: 100, salary: 13000 }, number_of_employees: 1 },    { _id: { dept_id: 200, salary: 14000 }, number_of_employees: 1 },    { _id: { dept_id: 300, salary: 25000 }, number_of_employees: 1 },    { _id: { dept_id: 400, salary: 15000 }, number_of_employees: 1 },    { _id: { dept_id: 300, salary: 20000 }, number_of_employees: 1 },    { _id: { dept_id: 100, salary: 10000 }, number_of_employees: 1 },    { _id: { dept_id: 100, salary: 12000 }, number_of_employees: 2 }  ]

The output shows a document for each unique pair ofdept_idandsalaryfield values. Hence, the documents are grouped according to the unique set of values in these fields.

Apart from grouping, we might also need to show the output data in a particular order. For this, we use the$sortstage in MongoDB aggregation pipelines.

The$sortStage

The$sortstage is used to sort the documents by given fields. We use the following syntax for the$sortstage in an aggregation pipeline;

db.collection_name.aggregate(  
[{  
  $sort:{  
    field_name1:1,  
    field_name2:-1
  }  
}]  
)

In the above syntax,

  • The document passed to the$sortstage contains field names from the dataset as keys with 1 or -1 as their values.
  • We set 1 as the value for a field name to sort the documents by the field in ascending order.
  • To sort the documents in descending order by a given field, we set the value to -1 for the field name.
  • When multiple field names are present in the document passed to the$sortstage, the output documents from thecollection_namecollection are sorted in the same order as the field names.
  • For the above syntax, the documents are first sorted by thefield_name1field in ascending order. If two documents have the same value in thefield_name1field, they are sorted byfield_name2field in descending order.

To understand this, we will sort the documents in theemployeescollection bysalaryin ascending order. To do this, we will pass the document{salary:1}to the$sortstage like so:

codecademy> db.employees.aggregate([{$sort: {salary:1}}])  
Copy to clipboard
codecademy> db.employees.aggregate([{$sort: {salary:1}}])

Output:

[    {      _id: ObjectId('6636483d3e7fbdb6982202d8'),      emp_id: 1,      emp_name: 'Ankit',      dept_id: 100,      salary: 10000,      emp_age: 32  },    {      _id: ObjectId('6636483d3e7fbdb6982202d9'),      emp_id: 2,      emp_name: 'Aditya',      dept_id: 100,      salary: 12000,      emp_age: 25    },    ....  ]  
Copy to clipboard
[    {      _id: ObjectId('6636483d3e7fbdb6982202d8'),      emp_id: 1,      emp_name: 'Ankit',      dept_id: 100,      salary: 10000,      emp_age: 32  },    {      _id: ObjectId('6636483d3e7fbdb6982202d9'),      emp_id: 2,      emp_name: 'Aditya',      dept_id: 100,      salary: 12000,      emp_age: 25    },    ....  ]

In the output, the documents are presented in the ascending order of their salaries.

We can also sort the documents by two field names. To sort the documents by thesalaryfield in ascending order and theagefield in descending order, we can pass the document{salary:1, age:-1}:

codecademy> db.employees.aggregate([{$sort: {salary:1, age:-1}}])  
Copy to clipboard
codecademy> db.employees.aggregate([{$sort: {salary:1, age:-1}}])
[    {      _id: ObjectId('6636483d3e7fbdb6982202d8'),      emp_id: 1,      emp_name: 'Ankit',      dept_id: 100,      salary: 10000,      emp_age: 32    },    {      _id: ObjectId('6636483d3e7fbdb6982202d9'),      emp_id: 2,      emp_name: 'Aditya',      dept_id: 100,      salary: 12000,      emp_age: 25    },    {      _id: ObjectId('6636483d3e7fbdb6982202db'),      emp_id: 4,      emp_name: 'Katy',      dept_id: 100,      salary: 12000,      emp_age: 23    },    ......    ]  
Copy to clipboard
[    {      _id: ObjectId('6636483d3e7fbdb6982202d8'),      emp_id: 1,      emp_name: 'Ankit',      dept_id: 100,      salary: 10000,      emp_age: 32    },    {      _id: ObjectId('6636483d3e7fbdb6982202d9'),      emp_id: 2,      emp_name: 'Aditya',      dept_id: 100,      salary: 12000,      emp_age: 25    },    {      _id: ObjectId('6636483d3e7fbdb6982202db'),      emp_id: 4,      emp_name: 'Katy',      dept_id: 100,      salary: 12000,      emp_age: 23    },    ......    ]

Note the documents are sorted by thesalaryfield in ascending order in the output. If two documents have the same value in thesalaryfield, they are sorted by theagefield in descending order.

Until now, the stages’ outputs contain all the fields in the documents. In most cases, we only want to access some fields stored in the data. In such situations, the$projectstage comes to our rescue.

The$projectStage

In a MongoDB aggregation pipeline, we use the$projectstage to retrieve selected fields from given documents. It has the following syntax:

db.collection_name.aggregate( 
[  
{  
  $project:{  
    field_name1:1,  
    field_name2:1  
  }  
}  
])

In the above syntax,

  • To retrieve a particular field from the documents, we set the value to 1 for the corresponding field name. To omit a particular field from the output, we set the value to 0.
  • If a field name isn’t mentioned in the document passed to the$projectstage, the field is not included in the output documents by default.

To understand this, let’s select the names of all the employees in theemployeescollection. For this, we will pass the document{emp_name:1}to the$projectstage, as shown below:

codecademy> db.employees.aggregate([{$project: {emp_name:1}}])  
Copy to clipboard
codecademy> db.employees.aggregate([{$project: {emp_name:1}}])
[    { _id: ObjectId('6636483d3e7fbdb6982202d8'), emp_name: 'Ankit' },    { _id: ObjectId('6636483d3e7fbdb6982202d9'), emp_name: 'Aditya' },    { _id: ObjectId('6636483d3e7fbdb6982202da'), emp_name: 'Adam' },    { _id: ObjectId('6636483d3e7fbdb6982202db'), emp_name: 'Katy' },    { _id: ObjectId('6636483d3e7fbdb6982202dc'), emp_name: 'Akshat' },    { _id: ObjectId('6636483d3e7fbdb6982202dd'), emp_name: 'Jasmine' },    { _id: ObjectId('6636483d3e7fbdb6982202de'), emp_name: 'Samantha' },    { _id: ObjectId('6636483d3e7fbdb6982202df'), emp_name: 'Cole' },    { _id: ObjectId('6636483d3e7fbdb6982202e0'), emp_name: 'Kunal' },    { _id: ObjectId('6636483d3e7fbdb6982202e1'), emp_name: 'Joe' }  ]  
Copy to clipboard
[    { _id: ObjectId('6636483d3e7fbdb6982202d8'), emp_name: 'Ankit' },    { _id: ObjectId('6636483d3e7fbdb6982202d9'), emp_name: 'Aditya' },    { _id: ObjectId('6636483d3e7fbdb6982202da'), emp_name: 'Adam' },    { _id: ObjectId('6636483d3e7fbdb6982202db'), emp_name: 'Katy' },    { _id: ObjectId('6636483d3e7fbdb6982202dc'), emp_name: 'Akshat' },    { _id: ObjectId('6636483d3e7fbdb6982202dd'), emp_name: 'Jasmine' },    { _id: ObjectId('6636483d3e7fbdb6982202de'), emp_name: 'Samantha' },    { _id: ObjectId('6636483d3e7fbdb6982202df'), emp_name: 'Cole' },    { _id: ObjectId('6636483d3e7fbdb6982202e0'), emp_name: 'Kunal' },    { _id: ObjectId('6636483d3e7fbdb6982202e1'), emp_name: 'Joe' }  ]

To select multiple fields, we can pass all the field names to the$projectstage. For instance, we can pass the document{emp_name:1, salary:1}to the$projectstage to select thenameandsalaryof the employees.

codecademy> db.employees.aggregate([{$project: {emp_name:1,salary:1}}])  
Copy to clipboard
codecademy> db.employees.aggregate([{$project: {emp_name:1,salary:1}}])
[    {      _id: ObjectId('6636483d3e7fbdb6982202d8'),      emp_name: 'Ankit',      salary: 10000    },    {      _id: ObjectId('6636483d3e7fbdb6982202d9'),      emp_name: 'Aditya',      salary: 12000    },    ......  ]  
Copy to clipboard
[    {      _id: ObjectId('6636483d3e7fbdb6982202d8'),      emp_name: 'Ankit',      salary: 10000    },    {      _id: ObjectId('6636483d3e7fbdb6982202d9'),      emp_name: 'Aditya',      salary: 12000    },    ......  ]

By default, we always get the document’s ID in the output. To exclude this, we can set the_idkey to 0 in the document passed to the$projectstage.

codecademy> db.employees.aggregate([{$project: {emp_name:1,salary:1,_id:0}}])  
Copy to clipboard
codecademy> db.employees.aggregate([{$project: {emp_name:1,salary:1,_id:0}}])

Notice that we don’t get the_idfield in the output after setting_idto 0:

[    { emp_name: 'Ankit', salary: 10000 },    { emp_name: 'Aditya', salary: 12000 },    { emp_name: 'Adam', salary: 13000 },    { emp_name: 'Katy', salary: 12000 },    { emp_name: 'Akshat', salary: 15000 },    { emp_name: 'Jasmine', salary: 14000 },    { emp_name: 'Samantha', salary: 16000 },    { emp_name: 'Cole', salary: 20000 },    { emp_name: 'Kunal', salary: 25000 },    { emp_name: 'Joe', salary: 15000 }  
]  
Copy to clipboard
[    { emp_name: 'Ankit', salary: 10000 },    { emp_name: 'Aditya', salary: 12000 },    { emp_name: 'Adam', salary: 13000 },    { emp_name: 'Katy', salary: 12000 },    { emp_name: 'Akshat', salary: 15000 },    { emp_name: 'Jasmine', salary: 14000 },    { emp_name: 'Samantha', salary: 16000 },    { emp_name: 'Cole', salary: 20000 },    { emp_name: 'Kunal', salary: 25000 },    { emp_name: 'Joe', salary: 15000 }  
]

Conclusion

Understanding the different stages in MongoDB aggregation pipelines can help you easily solve analytical questions. Here are some key takeaways from this article:

  • MongoDB aggregation pipelines consist of one or morestagesthat perform specific operations on documents.
  • A particular stage in the MongoDB pipeline filters data, groups documents, sorts them, or selects specific fields from the documents.
  • To combine different stages together and execute the MongoDB pipelines, we use theaggregate()function.

To learn more about solving analytical problems using the discussed concepts, you can read this article onMongoDB aggregation pipelines.

'The Codecademy Team, composed of experienced educators and tech experts, is dedicated to making tech skills accessible to all. We empower learners worldwide with expert-reviewed content that develops and enhances the technical skills needed to advance and succeed in their careers.'

30% Free Preview Reached · Unlock Remaining 70%

Complete Payment to Unlock Full Guide & Source Code

You are viewing the 30% preview. Join TopCourseAI Pro to unlock the remaining 70% of this article, unblur all code snippets, and gain full access to all 412 technical projects and guides.

✓ Full Access to All 160+ Projects
✓ Instant Blur Removal Everywhere
✓ Production Source Code & Blueprints
✓ All Future Deep Dives Included
Instant Unhindered Activation ·
End of Preview · Unlock Complete Blueprint & Source Code

Ready to access the complete step-by-step tutorial?

Gain full unhindered access to all 412 technical projects and guides, unblur all source code snippets, and download runnable scripts.

Production AI Blueprints

Ready to build production-grade AI systems?

Enjoyed this guide? Continue exploring our library of 166+ production blueprints, multi-agent swarms, and autonomous agent systems with 100% unblurred, executable source code.