| :mod:`airflow.contrib.hooks.mongo_hook` |
| ======================================= |
| |
| .. py:module:: airflow.contrib.hooks.mongo_hook |
| |
| |
| Module Contents |
| --------------- |
| |
| .. py:class:: MongoHook(conn_id='mongo_default', *args, **kwargs) |
| |
| Bases: :class:`airflow.hooks.base_hook.BaseHook` |
| |
| PyMongo Wrapper to Interact With Mongo Database |
| Mongo Connection Documentation |
| https://docs.mongodb.com/manual/reference/connection-string/index.html |
| You can specify connection string options in extra field of your connection |
| https://docs.mongodb.com/manual/reference/connection-string/index.html#connection-string-options |
| |
| If you want use DNS seedlist, set `srv` to True. |
| |
| ex. |
| {"srv": true, "replicaSet": "test", "ssl": true, "connectTimeoutMS": 30000} |
| |
| .. attribute:: conn_type |
| :annotation: = mongo |
| |
| |
| |
| |
| .. method:: __enter__(self) |
| |
| |
| |
| |
| .. method:: __exit__(self, exc_type, exc_val, exc_tb) |
| |
| |
| |
| |
| .. method:: get_conn(self) |
| |
| Fetches PyMongo Client |
| |
| |
| |
| |
| .. method:: close_conn(self) |
| |
| |
| |
| |
| .. method:: get_collection(self, mongo_collection, mongo_db=None) |
| |
| Fetches a mongo collection object for querying. |
| |
| Uses connection schema as DB unless specified. |
| |
| |
| |
| |
| .. method:: aggregate(self, mongo_collection, aggregate_query, mongo_db=None, **kwargs) |
| |
| Runs an aggregation pipeline and returns the results |
| https://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.aggregate |
| https://api.mongodb.com/python/current/examples/aggregation.html |
| |
| |
| |
| |
| .. method:: find(self, mongo_collection, query, find_one=False, mongo_db=None, **kwargs) |
| |
| Runs a mongo find query and returns the results |
| https://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.find |
| |
| |
| |
| |
| .. method:: insert_one(self, mongo_collection, doc, mongo_db=None, **kwargs) |
| |
| Inserts a single document into a mongo collection |
| https://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.insert_one |
| |
| |
| |
| |
| .. method:: insert_many(self, mongo_collection, docs, mongo_db=None, **kwargs) |
| |
| Inserts many docs into a mongo collection. |
| https://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.insert_many |
| |
| |
| |
| |
| .. method:: update_one(self, mongo_collection, filter_doc, update_doc, mongo_db=None, **kwargs) |
| |
| Updates a single document in a mongo collection. |
| https://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.update_one |
| |
| :param mongo_collection: The name of the collection to update. |
| :type mongo_collection: str |
| :param filter_doc: A query that matches the documents to update. |
| :type filter_doc: dict |
| :param update_doc: The modifications to apply. |
| :type update_doc: dict |
| :param mongo_db: The name of the database to use. |
| Can be omitted; then the database from the connection string is used. |
| :type mongo_db: str |
| |
| |
| |
| |
| .. method:: update_many(self, mongo_collection, filter_doc, update_doc, mongo_db=None, **kwargs) |
| |
| Updates one or more documents in a mongo collection. |
| https://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.update_many |
| |
| :param mongo_collection: The name of the collection to update. |
| :type mongo_collection: str |
| :param filter_doc: A query that matches the documents to update. |
| :type filter_doc: dict |
| :param update_doc: The modifications to apply. |
| :type update_doc: dict |
| :param mongo_db: The name of the database to use. |
| Can be omitted; then the database from the connection string is used. |
| :type mongo_db: str |
| |
| |
| |
| |
| .. method:: replace_one(self, mongo_collection, doc, filter_doc=None, mongo_db=None, **kwargs) |
| |
| Replaces a single document in a mongo collection. |
| https://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.replace_one |
| |
| .. note:: |
| If no ``filter_doc`` is given, it is assumed that the replacement |
| document contain the ``_id`` field which is then used as filters. |
| |
| :param mongo_collection: The name of the collection to update. |
| :type mongo_collection: str |
| :param doc: The new document. |
| :type doc: dict |
| :param filter_doc: A query that matches the documents to replace. |
| Can be omitted; then the _id field from doc will be used. |
| :type filter_doc: dict |
| :param mongo_db: The name of the database to use. |
| Can be omitted; then the database from the connection string is used. |
| :type mongo_db: str |
| |
| |
| |
| |
| .. method:: replace_many(self, mongo_collection, docs, filter_docs=None, mongo_db=None, upsert=False, collation=None, **kwargs) |
| |
| Replaces many documents in a mongo collection. |
| |
| Uses bulk_write with multiple ReplaceOne operations |
| https://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.bulk_write |
| |
| .. note:: |
| If no ``filter_docs``are given, it is assumed that all |
| replacement documents contain the ``_id`` field which are then |
| used as filters. |
| |
| :param mongo_collection: The name of the collection to update. |
| :type mongo_collection: str |
| :param docs: The new documents. |
| :type docs: list[dict] |
| :param filter_docs: A list of queries that match the documents to replace. |
| Can be omitted; then the _id fields from docs will be used. |
| :type filter_docs: list[dict] |
| :param mongo_db: The name of the database to use. |
| Can be omitted; then the database from the connection string is used. |
| :type mongo_db: str |
| :param upsert: If ``True``, perform an insert if no documents |
| match the filters for the replace operation. |
| :type upsert: bool |
| :param collation: An instance of |
| :class:`~pymongo.collation.Collation`. This option is only |
| supported on MongoDB 3.4 and above. |
| :type collation: pymongo.collation.Collation |
| |
| |
| |
| |
| .. method:: delete_one(self, mongo_collection, filter_doc, mongo_db=None, **kwargs) |
| |
| Deletes a single document in a mongo collection. |
| https://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.delete_one |
| |
| :param mongo_collection: The name of the collection to delete from. |
| :type mongo_collection: str |
| :param filter_doc: A query that matches the document to delete. |
| :type filter_doc: dict |
| :param mongo_db: The name of the database to use. |
| Can be omitted; then the database from the connection string is used. |
| :type mongo_db: str |
| |
| |
| |
| |
| .. method:: delete_many(self, mongo_collection, filter_doc, mongo_db=None, **kwargs) |
| |
| Deletes one or more documents in a mongo collection. |
| https://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.delete_many |
| |
| :param mongo_collection: The name of the collection to delete from. |
| :type mongo_collection: str |
| :param filter_doc: A query that matches the documents to delete. |
| :type filter_doc: dict |
| :param mongo_db: The name of the database to use. |
| Can be omitted; then the database from the connection string is used. |
| :type mongo_db: str |
| |
| |
| |
| |