The driver and tips here will work with any Mongo compatible database. The result is the difference between queries lasting seconds or minutes and taking only milliseconds to complete. The ORM maps a relational entity to a domain entity, whereas the solution I detailed here maps a dynamic document-based entity to a strongly-typed domain entity. In a way, object-relational mappers (ORM) tools like Entity Framework Core provide a similar solution. The solution wasn’t to keep hammering at the query, but instead to take advantage of the nature of the document database and store the documents differently. I spent a day trying to optimize the sub-query that scanned the nutrients by amount before I realized I was approaching a document database problem with a relational-database mindset. Putting it all TogetherĪs developers it’s important to recognize when we are trying to solve the wrong problem. The refactored query returns results in milliseconds instead of timing out as it did before. Regardless of the property name you use for the unique identifier in your C# model, the database will always internally store the identifier property with the name “_id”. Note that the only difference in the results is that each item is a “BsonDocument” instead of a “FoodItem” but can easily be cast/transformed from one to the other. My first pass at a class definition looked like this: Some foods in the database only have a few entries for nutrient information, while others may have several hundred values. cooked) and a list of nutrient information. Each food item has one or more weights or “units of measure” associated with it (think: tablespoons vs. The key component of the USDA database that I am interested in is the individual food items. I built an application to import them into CosmosDB that I’ll share in a follow-up post after I present my CosmosDB talk. The files are well-documented and relatively easy to parse. This database contains lists of foods, food groups, and other nutrient data that you can query for information like protein content and total calories using different weights and measures. To keep things simple and provide a meaningful demo, I chose to work with the freely available USDA National Nutrient Database for Standard Reference. Believe it or not, it’s possible to have the best of both worlds.įood Item definition in a. Paradoxically, as a JavaScript developer I love the dynamic flexibility that JSON documents provide. As a C# developer, I love that I can create strongly-typed domain objects and work with “known entities” as part of my application. I chose the MongoDB driver due to its popularity and because I’m already familiar with it through my Node.js work. NET Core 2.0 project to demonstrate several features of CosmosDB. If you don’t have one, here’s an instruction on how to set that up.Recently I created a. I’m going to use MongoDB’s standard sample data called ‘restaurants’ for most of the part. There are a few things you need to know before proceeding further. I’m going to use Exploratory’s UI to demonstrate, but all the examples can be used with any other tools including Mongo Shell. Today, I’m going to walk you through the basics of MongoDB query with a bunch of examples so that you can start writing the queries to work with MongoDB data effectively. Though you don’t need to be an expert of MongoDB query, knowing just the enough level of MongoDB query will make your entire data wrangling and analysis flow much more smooth and more efficient. So I’d usually recommend a hybrid approach, which is to use MongoDB Query at a minimal level to extract just the enough data, not the whole, from the database and import it into Exploratory, then do the nitty gritty world of data wrangling there for faster and easier data exploration. However, as you would imagine, that might not be always efficient or even practical sometimes due to the time it takes to download the data and the memory size limit of your PC. or simply with UI tools like Exploratory (UI for R). Well, of course, you can always import the whole data from MongoDB to R’s in-memory for much easier data exploration with packages like ‘mongolite’, ‘dplyr’, ‘tidyr’, etc. Now having said that, querying MongoDB data is not as straightforward as dplyr or even SQL due to the MongoDB Query’s JSON and JavaScript flavor that makes the queries hard to read and a bit more complicated. We store most of the data we have in MongoDB because it’s so easy to store and manage in a way that allows us to keep changing the scheme, which is super critical and essential for startups like us who are constantly experimenting by changing things. We at Exploratory use MongoDB quite a lot.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |