diff --git a/README.rst b/README.rst index 071e1dd4..3a36f326 100644 --- a/README.rst +++ b/README.rst @@ -56,7 +56,7 @@ Re-build the openedx-dev image and launch the platform:: Administration ************** -Deployment of the forum v2 application is gated by two course waffle flags. In addition, this application provides a few commands to facilitate the transition from the legacy forum app. +This application provides a few commands to facilitate the transition from the legacy forum app. Forum v2 toggle --------------- @@ -67,34 +67,10 @@ In edx-platform, forum v2 is not enabled by default and edx-platform will keep c Note that Tutor enables this flag for all forum plugin users, such that you don't have to run this command yourself. If you wish to migrate your courses one by one to the new forum v2 app, you may create the corresponding "Waffle flag course override" objects in your LMS administration panel, at: ``http(s):///admin/waffle_utils/waffleflagcourseoverridemodel/``. -⚠️⚠️⚠️ Even if the forum v2 toggle is not enabled, edx-platform will make a call to the forum v2 API in some edge cases. That's because edx-platform needs to determine whether it should use forum v2 or cs_comments_service, based on the value of some course waffle flag. In order to access the course wafffle flag, we need to determine the course ID of the current HTTP request. In some requests, the course ID is not available: only the thread ID or the comment ID is. Thus, edx-platform needs to fetch the course ID that is associated to the thread or comment. That information is stored either in MySQL or in MongoDB. Thus, edx-platform needs to call the forum v2 API. - -As a consequence, **the forum v2 app needs to have accurate MongoDB configuration settings even if you don't use forum v2**. In a Tutor installation, these settings are set to the right values. In other environments, the following Django settings must be set:: - - # Name of the MongoDB database in which forum data is stored - FORUM_MONGODB_DATABASE = "cs_comments_service" - - # This setting will be passed to the MongoDB client constructor as follows: - # pymongo.MongoClient(**FORUM_MONGODB_CLIENT_PARAMETERS) - # Documentation: https://pymongo.readthedocs.io/en/4.4.0/api/pymongo/mongo_client.html#pymongo.mongo_client.MongoClient - FORUM_MONGODB_CLIENT_PARAMETERS = {"host": "mongodb"} - -MySQL backend toggle --------------------- - -To preserve the legacy behaviour of storing data in MongoDB, the forum v2 app makes it possible to keep using MongoDB as a data backend. However, it is strongly recommended to switch to the MySQL storage backend by toggling the ``forum_v2.enable_mysql_backend`` course waffle flag:: - - ./manage.py lms waffle_flag --create --everyone forum_v2.enable_mysql_backend - -Here again, Tutor creates this flag by default, such that you don't have to create it yourself. If you decide to switch to MySQL, you will have to migrate your data from MongoDB -- see instructions below. - Migration from MongoDB to MySQL ------------------------------- -The forum v2 app comes with the ``forum_migrate_courses_to_mysql`` migration command to move data from MongoDB to MySQL. This command will perform the following steps: - -1. Migrate data: user, content and read state data from MongoDB to MySQL. -2. Enable the ``forum_v2.enable_mysql_backend`` waffle flag for the specified course(s). +The forum v2 app comes with the ``forum_migrate_course_from_mongodb_to_mysql`` migration command to move data from MongoDB to MySQL. This command migrates user, content, and read state data from MongoDB to MySQL. To migrate data for specific courses, run the command with the course IDs as argument:: @@ -104,16 +80,10 @@ To migrate data for all courses, run the command with the ``all`` argument:: ./manage.py lms forum_migrate_course_from_mongodb_to_mysql all -To test data migration without actually creating course toggles, use the ``--no-toggle`` option:: - - ./manage.py lms forum_migrate_course_from_mongodb_to_mysql --no-toggle all - -⚠️ Note that the command will create toggles only for the processed courses. Courses created in the future will not automatically use the MySQL backend unless you create the global waffle flag with the ``waffle_flag --create`` command indicated above. - MongoDB data deletion --------------------- -After you have successfully migrated your course data from MySQL to MongoDB using the command above, you may delete your MongoDB data using the ``forum_delete_course_from_mongodb`` management command. This command deletes course data from MongoDB for the specified courses. +After you have successfully migrated your course data from MongoDB to MySQL using the command above, you may delete your MongoDB data using the ``forum_delete_course_from_mongodb`` management command. This command deletes course data from MongoDB for the specified courses. Run the command with the course ID(s) as an argument:: @@ -127,15 +97,6 @@ To try out changes before applying them, use the ``--dry-run`` option. For insta ./manage.py lms forum_delete_course_from_mongodb all --dry-run -MongoDB Indexes ---------------- - -To optimize MongoDB query performance, it is crucial to create database indexes. The command will create or update indexes and skip them if they already exist. - -To create or update MongoDB indexes, execute the following command:: - - ./manage.py lms forum_create_mongodb_indexes - Search Indicies --------------- diff --git a/forum/__init__.py b/forum/__init__.py index 1257036d..15fb549a 100644 --- a/forum/__init__.py +++ b/forum/__init__.py @@ -2,4 +2,4 @@ Openedx forum app. """ -__version__ = "0.4.2" +__version__ = "0.4.3" diff --git a/forum/api/comments.py b/forum/api/comments.py index edc14a1c..c95a1d4b 100644 --- a/forum/api/comments.py +++ b/forum/api/comments.py @@ -10,6 +10,7 @@ from rest_framework.serializers import ValidationError from forum.backend import get_backend +from forum.backends.mysql.api import MySQLBackend from forum.serializers.comment import CommentSerializer from forum.utils import ForumV2RequestError @@ -307,17 +308,8 @@ def create_parent_comment( def get_course_id_by_comment(comment_id: str) -> str | None: """ Return course_id for the matching comment. - It searches for comment_id both in mongodb and mysql. """ - # pylint: disable=C0415 - from forum.backends.mongodb.api import MongoBackend - from forum.backends.mysql.api import MySQLBackend - - return ( - MongoBackend.get_course_id_by_comment_id(comment_id) - or MySQLBackend.get_course_id_by_comment_id(comment_id) - or None - ) + return MySQLBackend.get_course_id_by_comment_id(comment_id) or None def get_user_comments( diff --git a/forum/api/threads.py b/forum/api/threads.py index f04bcc80..fc089758 100644 --- a/forum/api/threads.py +++ b/forum/api/threads.py @@ -10,6 +10,7 @@ from forum.api.users import mark_thread_as_read from forum.backend import get_backend +from forum.backends.mysql.api import MySQLBackend from forum.serializers.thread import ThreadSerializer from forum.utils import ForumV2RequestError, get_int_value_from_collection, str_to_bool @@ -402,14 +403,5 @@ def get_user_threads( def get_course_id_by_thread(thread_id: str) -> str | None: """ Return course_id for the matching thread. - It searches for thread_id both in mongodb and mysql. """ - # pylint: disable=C0415 - from forum.backends.mongodb.api import MongoBackend - from forum.backends.mysql.api import MySQLBackend - - return ( - MongoBackend.get_course_id_by_thread_id(thread_id) - or MySQLBackend.get_course_id_by_thread_id(thread_id) - or None - ) + return MySQLBackend.get_course_id_by_thread_id(thread_id) or None diff --git a/forum/backend.py b/forum/backend.py index bc2434dc..99a70dc0 100644 --- a/forum/backend.py +++ b/forum/backend.py @@ -1,43 +1,10 @@ """Backend module for forum.""" -from typing import Callable, Optional - -from forum.backends.mongodb.api import MongoBackend from forum.backends.mysql.api import MySQLBackend -def is_mysql_backend_enabled(course_id: str | None) -> bool: - """ - Return True if mysql backend is enabled for the course. - """ - try: - # pylint: disable=import-outside-toplevel - from forum.toggles import ENABLE_MYSQL_BACKEND - from opaque_keys import InvalidKeyError - from opaque_keys.edx.keys import CourseKey - except ImportError: - return True - - course_key: "CourseKey" | None = None - if isinstance(course_id, CourseKey): - course_key = course_id # type: ignore[unreachable] - elif isinstance(course_id, str): - try: - course_key = CourseKey.from_string(course_id) - except InvalidKeyError: - pass - - return ENABLE_MYSQL_BACKEND.is_enabled(course_key) - - -def get_backend( - course_id: Optional[str] = None, -) -> Callable[[], MongoBackend | MySQLBackend]: - """Return a factory function that lazily loads the backend API based on course_id.""" - - def _get_backend() -> MongoBackend | MySQLBackend: - if is_mysql_backend_enabled(course_id): - return MySQLBackend() - return MongoBackend() - - return _get_backend +def get_backend( # pylint: disable=unused-argument + course_id: str | None = None, +) -> "type[MySQLBackend]": + """Return the MySQL backend.""" + return MySQLBackend diff --git a/forum/backends/mongodb/__init__.py b/forum/backends/mongodb/__init__.py deleted file mode 100644 index 569ce740..00000000 --- a/forum/backends/mongodb/__init__.py +++ /dev/null @@ -1,21 +0,0 @@ -""" -Mongo Models -""" - -from .comments import Comment -from .contents import BaseContents, Contents -from .subscriptions import Subscriptions -from .threads import CommentThread -from .users import Users - -__all__ = [ - "BaseContents", - "Comment", - "Contents", - "CommentThread", - "Subscriptions", - "Users", - "MODEL_INDICES", -] - -MODEL_INDICES: tuple[type[BaseContents], ...] = (CommentThread, Comment) diff --git a/forum/backends/mongodb/api.py b/forum/backends/mongodb/api.py deleted file mode 100644 index 01124532..00000000 --- a/forum/backends/mongodb/api.py +++ /dev/null @@ -1,1795 +0,0 @@ -"""Model util function for db operations.""" - -import math -from datetime import datetime, timezone -from typing import Any, Optional - -from bson import ObjectId, errors as bson_errors -from django.core.exceptions import ObjectDoesNotExist - -from forum.backends.backend import AbstractBackend -from forum.backends.mongodb import ( - Comment, - CommentThread, - Contents, - Subscriptions, - Users, -) -from forum.constants import RETIRED_BODY, RETIRED_TITLE -from forum.utils import ( - ForumV2RequestError, - get_group_ids_from_params, - get_sort_criteria, - make_aware, - str_to_bool, -) - - -class MongoBackend(AbstractBackend): - """Mongodb Backend API.""" - - @classmethod - def update_stats_for_course( - cls, user_id: str, course_id: str, **kwargs: Any - ) -> None: - """Update stats for a course.""" - user = Users().get(user_id) - if not user: - return - course_stats = user.get("course_stats", []) - for course_stat in course_stats: - if course_stat["course_id"] == course_id: - course_stat.update( - { - k: course_stat[k] + v - for k, v in kwargs.items() - if k in course_stat - } - ) - Users().update( - user_id, - course_stats=course_stats, - ) - return - cls.build_course_stats(user["_id"], course_id) - - @classmethod - def flag_as_abuse( - cls, user_id: str, entity_id: str, **kwargs: Any - ) -> dict[str, Any]: - """ - Flag an entity as abuse. - - Args: - user (dict[str, Any]): The user who is flagging the entity as abuse. - entity (dict[str, Any]): The entity being flagged as abuse. - - Returns: - dict[str, Any]: The updated entity with the abuse flag. - - Raises: - ValueError: If user ID or entity is not provided. - """ - user = Users().get(user_id) - entity = Contents().get(entity_id) - if not (user and entity): - raise ValueError("User ID or entity is not provided") - abuse_flaggers = entity["abuse_flaggers"] - first_flag_added = False - if user["_id"] not in abuse_flaggers: - abuse_flaggers.append(user["_id"]) - first_flag_added = len(abuse_flaggers) == 1 - Contents().update( - entity["_id"], - abuse_flaggers=abuse_flaggers, - ) - if first_flag_added: - cls.update_stats_for_course( - entity["author_id"], - entity["course_id"], - active_flags=1, - ) - updated_content = Contents().get(entity["_id"]) - if not updated_content: - raise ValueError("Entity not found") - return updated_content - - @classmethod - def update_stats_after_unflag( - cls, user_id: str, entity_id: str, has_no_historical_flags: bool, **kwargs: Any - ) -> None: - """Update the stats for the course after unflagging an entity.""" - entity = Contents().get(entity_id) - if not entity: - raise ObjectDoesNotExist - - first_historical_flag = ( - has_no_historical_flags and not entity["historical_abuse_flaggers"] - ) - if first_historical_flag: - cls.update_stats_for_course(user_id, entity["course_id"], inactive_flags=1) - - if not entity["abuse_flaggers"]: - cls.update_stats_for_course(user_id, entity["course_id"], active_flags=-1) - - @classmethod - def un_flag_as_abuse( - cls, user_id: str, entity_id: str, **kwargs: Any - ) -> dict[str, Any]: - """ - Unflag an entity as abuse. - - Args: - user (dict[str, Any]): The user who is unflagging the entity as abuse. - entity (dict[str, Any]): The entity being unflagged as abuse. - - Returns: - dict[str, Any]: The updated entity with the abuse flag removed. - - Raises: - ValueError: If user ID or entity is not provided. - """ - user = Users().get(user_id) - entity = Contents().get(entity_id) - if not (user and entity): - raise ValueError("User ID or entity is not provided") - - has_no_historical_flags = len(entity["historical_abuse_flaggers"]) == 0 - if user["_id"] in entity["abuse_flaggers"]: - entity["abuse_flaggers"].remove(user["_id"]) - Contents().update( - entity["_id"], - abuse_flaggers=entity["abuse_flaggers"], - ) - cls.update_stats_after_unflag( - entity["author_id"], entity["_id"], has_no_historical_flags - ) - updated_content = Contents().get(entity["_id"]) - if not updated_content: - raise ValueError("Entity not found") - return updated_content - - @classmethod - def un_flag_all_as_abuse(cls, entity_id: str, **kwargs: Any) -> dict[str, Any]: - """ - Unflag an entity as abuse for all users. - - Args: - entity (dict[str, Any]): The entity being unflagged as abuse. - - Returns: - dict[str, Any]: The updated entity with all abuse flags removed. - - Raises: - ValueError: If entity is not provided. - """ - entity = Contents().get(entity_id) - if not entity: - raise ValueError("Entity is not provided") - has_no_historical_flags = len(entity["historical_abuse_flaggers"]) == 0 - historical_abuse_flaggers = list( - set(entity["historical_abuse_flaggers"]) | set(entity["abuse_flaggers"]) - ) - Contents().update( - entity["_id"], - abuse_flaggers=[], - historical_abuse_flaggers=historical_abuse_flaggers, - ) - cls.update_stats_after_unflag( - entity["author_id"], entity["_id"], has_no_historical_flags - ) - updated_content = Contents().get(entity["_id"]) - if not updated_content: - raise ValueError("Entity not found") - return updated_content - - @staticmethod - def update_vote( - content_id: str, - user_id: str, - vote_type: str = "", - is_deleted: bool = False, - **kwargs: Any, - ) -> bool: - """ - Update a vote on a thread (either upvote or downvote). - - :param content: The content document containing vote data. - :param user: The user document for the user voting. - :param vote_type: String indicating the type of vote ('up' or 'down'). - :param is_deleted: Boolean indicating if the user is removing their vote (True) or voting (False). - :return: True if the vote was successfully updated, False otherwise. - """ - user = Users().get(user_id) - content = Contents().get(content_id) - if not (user and content): - raise ValueError("User ID or entity is not provided") - - votes: dict[str, Any] = content["votes"] - update_needed: bool = False - - if not is_deleted: - if vote_type not in ["up", "down"]: - raise ValueError("Invalid vote_type, use ('up' or 'down')") - - if vote_type == "up": - current_votes = set(votes["up"]) - opposite_votes = set(votes["down"]) - else: - current_votes = set(votes["down"]) - opposite_votes = set(votes["up"]) - - # Check if user is voting - if user_id not in current_votes: - current_votes.add(user_id) - update_needed = True - if user_id in opposite_votes: - opposite_votes.remove(user_id) - - updated_up_votes = opposite_votes if vote_type == "down" else current_votes - updated_down_votes = ( - current_votes if vote_type == "down" else opposite_votes - ) - - else: - # Handle vote deletion - updated_up_votes = set(votes["up"]) - updated_down_votes = set(votes["down"]) - - if user_id in updated_up_votes: - updated_up_votes.remove(user_id) - update_needed = True - if user_id in updated_down_votes: - updated_down_votes.remove(user_id) - update_needed = True - - if update_needed: - # Prepare updated votes - content_model = Contents() - updated_votes = content_model.get_votes_dict( - list(updated_up_votes), list(updated_down_votes) - ) - updated_count = content_model.update_votes( - content_id=content_id, votes=updated_votes - ) - return bool(updated_count) - - return False - - @classmethod - def upvote_content(cls, entity_id: str, user_id: str, **kwargs: Any) -> bool: - """ - Upvotes the specified thread or comment by the given user. - - Args: - thread (dict): The thread or comment data to be upvoted. - user (dict): The user who is performing the upvote. - - Returns: - bool: True if the vote was successfully updated, False otherwise. - """ - user = Users().get(user_id) - entity = Contents().get(entity_id) - if not (user and entity): - raise ValueError("User ID or entity is not provided") - - return cls.update_vote(entity["_id"], user["external_id"], vote_type="up") - - @classmethod - def downvote_content(cls, entity_id: str, user_id: str, **kwargs: Any) -> bool: - """ - Downvotes the specified thread or comment by the given user. - - Args: - thread (dict): The thread or comment data to be downvoted. - user (dict): The user who is performing the downvote. - - Returns: - bool: True if the vote was successfully updated, False otherwise. - """ - user = Users().get(user_id) - entity = Contents().get(entity_id) - if not (user and entity): - raise ValueError("User ID or entity is not provided") - - return cls.update_vote(entity["_id"], user["external_id"], vote_type="down") - - @classmethod - def remove_vote(cls, entity_id: str, user_id: str, **kwargs: Any) -> bool: - """ - Remove the vote (upvote or downvote) from the specified thread or comment for the given user. - - Args: - thread (dict): The thread or comment data from which the vote should be removed. - user (dict): The user who is removing their vote. - - Returns: - bool: True if the vote was successfully removed, False otherwise. - """ - user = Users().get(user_id) - entity = Contents().get(entity_id) - if not (user and entity): - raise ValueError("User ID or entity is not provided") - - return cls.update_vote(entity["_id"], user["external_id"], is_deleted=True) - - @staticmethod - def validate_thread_and_user( - user_id: str, thread_id: str - ) -> tuple[dict[str, Any], dict[str, Any]]: - """ - Validate thread and user. - - Arguments: - user_id (str): The ID of the user making the request. - thread_id (str): The ID of the thread. - - Returns: - tuple[dict[str, Any], dict[str, Any]]: A tuple containing the user and thread data. - - Raises: - ValueError: If the thread or user is not found. - """ - thread = CommentThread().get(thread_id) - user = Users().get(user_id) - if not (thread and user): - raise ValueError("User / Thread doesn't exist") - - return user, thread - - @staticmethod - def pin_unpin_thread(thread_id: str, action: str) -> None: - """ - Pin or unpin the thread based on action parameter. - - Arguments: - thread_id (str): The ID of the thread to pin/unpin. - action (str): The action to perform ("pin" or "unpin"). - """ - CommentThread().update( - thread_id, pinned=action == "pin", skip_timestamp_update=True - ) - - @classmethod - def get_pinned_unpinned_thread_serialized_data( - cls, user_id: str, thread_id: str, serializer_class: Any - ) -> dict[str, Any]: - """ - Return serialized data of pinned or unpinned thread. - - Arguments: - user (dict[str, Any]): The user who requested the action. - thread_id (str): The ID of the thread to pin/unpin. - - Returns: - dict[str, Any]: The serialized data of the pinned/unpinned thread. - - Raises: - ValueError: If the serialization is not valid. - """ - user = Users().get(user_id) - updated_thread = CommentThread().get(thread_id) - if not (user and updated_thread): - raise ValueError("User ID or entity is not provided") - - context = { - "user_id": user["_id"], - "username": user["username"], - "type": "thread", - "id": thread_id, - } - if updated_thread is not None: - context = {**context, **updated_thread} - serializer = serializer_class(data=context, backend=cls) - if not serializer.is_valid(): - raise ValueError(serializer.errors) - - return serializer.data - - @classmethod - def handle_pin_unpin_thread_request( - cls, user_id: str, thread_id: str, action: str, serializer_class: Any - ) -> dict[str, Any]: - """ - Catches pin/unpin thread request. - - - validates thread and user. - - pin or unpin the thread based on action parameter. - - return serialized data of thread. - - Arguments: - user_id (str): The ID of the user making the request. - thread_id (str): The ID of the thread to pin/unpin. - action (str): The action to perform ("pin" or "unpin"). - - Returns: - dict[str, Any]: The serialized data of the pinned/unpinned thread. - """ - user, _ = cls.validate_thread_and_user(user_id, thread_id) - cls.pin_unpin_thread(thread_id, action) - return cls.get_pinned_unpinned_thread_serialized_data( - user["external_id"], thread_id, serializer_class - ) - - @staticmethod - def get_abuse_flagged_count(thread_ids: list[str]) -> dict[str, int]: - """ - Retrieves the count of abuse-flagged comments for each thread in the provided list of thread IDs. - - Args: - thread_ids (list[str]): List of thread IDs to check for abuse flags. - - Returns: - dict[str, int]: A dictionary mapping thread IDs to their corresponding abuse-flagged comment count. - """ - pipeline: list[dict[str, Any]] = [ - { - "$match": { - "comment_thread_id": {"$in": [ObjectId(tid) for tid in thread_ids]}, - "abuse_flaggers": {"$ne": []}, - } - }, - {"$group": {"_id": "$comment_thread_id", "flagged_count": {"$sum": 1}}}, - ] - flagged_threads = Contents().aggregate(pipeline) - - return {str(item["_id"]): item["flagged_count"] for item in flagged_threads} - - @staticmethod - def get_read_states( - thread_ids: list[str], user_id: str, course_id: str - ) -> dict[str, list[Any]]: - """ - Retrieves the read state and unread comment count for each thread in the provided list. - - Args: - threads (list[dict[str, Any]]): list of threads to check read state for. - user_id (str): The ID of the user whose read states are being retrieved. - course_id (str): The course ID associated with the threads. - - Returns: - dict[str, list[Any]]: A dictionary mapping thread IDs to a list containing - whether the thread is read and the unread comment count. - """ - threads = CommentThread().find( - {"_id": {"$in": [ObjectId(thread_id) for thread_id in thread_ids]}} - ) - read_states = {} - if user_id: - user = Users().find_one( - {"_id": user_id, "read_states.course_id": course_id} - ) - if not user: - return {} - - read_state = next( - ( - rs - for rs in user.get("read_states", []) - if rs.get("course_id") == course_id - ), - None, - ) - if read_state: - read_dates = read_state.get("last_read_times", {}) - for thread in threads: - thread_key = str(thread["_id"]) - if thread_key in read_dates: - read_date = make_aware(read_dates[thread_key]) - last_activity_at = make_aware(thread["last_activity_at"]) - is_read = read_date >= last_activity_at - unread_comment_count = Contents().count_documents( - { - "comment_thread_id": ObjectId(thread_key), - "created_at": {"$gte": read_dates[thread_key]}, - "author_id": {"$ne": str(user_id)}, - } - ) - read_states[thread_key] = [is_read, unread_comment_count] - - return read_states - - @staticmethod - def get_endorsed(thread_ids: list[str]) -> dict[str, bool]: - """ - Retrieves endorsed status for each thread in the provided list of thread IDs. - - Args: - thread_ids (list[str]): List of thread IDs to check for endorsement. - - Returns: - dict[str, bool]: A dictionary of thread IDs to their endorsed status (True if endorsed, False otherwise). - """ - endorsed_comments = Comment().find( - { - "comment_thread_id": {"$in": [ObjectId(tid) for tid in thread_ids]}, - "endorsed": True, - } - ) - - return {str(item["comment_thread_id"]): True for item in endorsed_comments} - - @staticmethod - def get_user_read_state_by_course_id( - user_id: str, course_id: str - ) -> dict[str, Any]: - """ - Retrieves the user's read state for a specific course. - - Args: - user (dict[str, Any]): The user object containing read states. - course_id (str): The course ID to filter the user's read state by. - - Returns: - dict[str, Any]: The user's read state for the specified course, or an empty dictionary if not found. - """ - user = Users().get(user_id) - if not user: - raise ValueError("User does not exist.") - - for read_state in user.get("read_states", []): - if read_state["course_id"] == course_id: - return read_state - return {} - - # TODO: Make this function modular - # pylint: disable=too-many-nested-blocks,too-many-statements - @classmethod - def handle_threads_query( - cls, - comment_thread_ids: list[str], - user_id: str, - course_id: str, - group_ids: list[int], - author_id: Optional[str], - thread_type: Optional[str], - filter_flagged: bool, - filter_unread: bool, - filter_unanswered: bool, - filter_unresponded: bool, - count_flagged: bool, - sort_key: str, - page: int, - per_page: int, - context: str = "course", - raw_query: bool = False, - commentable_ids: Optional[list[str]] = None, - is_moderator: bool = False, - ) -> dict[str, Any]: - """ - Handles complex thread queries based on various filters and returns paginated results. - - Args: - comment_thread_ids (list[str]): List of comment thread IDs to filter. - user_id (str): The ID of the user making the request. - course_id (str): The course ID associated with the threads. - group_ids (list[int]): List of group IDs for group-based filtering. - author_id (str): The ID of the author to filter threads by. - thread_type (str): The type of thread to filter by. - filter_flagged (bool): Whether to filter threads flagged for abuse. - filter_unread (bool): Whether to filter unread threads. - filter_unanswered (bool): Whether to filter unanswered questions. - filter_unresponded (bool): Whether to filter threads with no responses. - count_flagged (bool): Whether to include flagged content count. - sort_key (str): The key to sort the threads by. - page (int): The page number for pagination. - per_page (int): The number of threads per page. - context (str): The context to filter threads by. - raw_query (bool): Whether to return raw query results without further processing. - commentable_ids (Optional[list[str]]): List of commentable IDs to filter threads by topic id. - is_moderator (bool): Whether the user is a discussion moderator. - - Returns: - dict[str, Any]: A dictionary containing the paginated thread results and associated metadata. - """ - # Convert thread_ids to ObjectId - comment_thread_obj_ids: list[ObjectId] = [] - - for tid in comment_thread_ids: - try: - thread_id = ObjectId(tid) - comment_thread_obj_ids.append(thread_id) - except bson_errors.InvalidId: - continue - - # Base query - base_query: dict[str, Any] = { - "_id": {"$in": comment_thread_obj_ids}, - "context": context, - } - - # Group filtering - if group_ids: - base_query["$or"] = [ - {"group_id": {"$in": group_ids}}, - {"group_id": {"$exists": False}}, - ] - - # Author filtering - if author_id: - base_query["author_id"] = author_id - if author_id != user_id: - base_query["anonymous"] = False - base_query["anonymous_to_peers"] = False - - # Thread type filtering - if thread_type: - base_query["thread_type"] = thread_type - - # Flagged content filtering - if filter_flagged: - flagged_query = { - "course_id": course_id, - "abuse_flaggers": {"$ne": [], "$exists": True}, - } - flagged_comments = Comment().distinct("comment_thread_id", flagged_query) - flagged_threads = CommentThread().distinct("_id", flagged_query) - base_query["_id"]["$in"] = list( - set(comment_thread_obj_ids) & set(flagged_comments + flagged_threads) - ) - - # Unanswered questions filtering - if filter_unanswered: - endorsed_threads = Comment().distinct( - "comment_thread_id", - { - "course_id": course_id, - "parent_id": {"$exists": False}, - "endorsed": True, - }, - ) - base_query["thread_type"] = "question" - base_query["_id"]["$nin"] = endorsed_threads - - # Unresponded threads filtering - if filter_unresponded: - base_query["comment_count"] = 0 - - # filter by topics: if commentable_ids are provided, commentable_id is basically topic id - # For moderators: show all topics (no filtering by commentable_ids) - # For learners: apply commentable_ids filtering (cohorted topics shown as archived) - if commentable_ids and not is_moderator: - base_query["commentable_id"] = {"$in": commentable_ids} - sort_criteria = get_sort_criteria(sort_key) - - comment_threads = CommentThread().find(base_query) - thread_count = CommentThread().count_documents(base_query) - - if sort_criteria or raw_query: - request_user = Users().get(user_id) if user_id else None - - if not raw_query: - comment_threads = comment_threads.sort(sort_criteria) - - if filter_unread and request_user: - read_state = cls.get_user_read_state_by_course_id( - request_user["external_id"], course_id - ) - read_dates = read_state.get("last_read_times", {}) - - threads = [] - skipped = 0 - to_skip = (page - 1) * per_page - has_more = False - batch_size = 100 - - for thread in comment_threads.batch_size(batch_size): - thread_key = str(thread["_id"]) - if ( - thread_key not in read_dates - or read_dates[thread_key] < thread["last_activity_at"] - ): - if raw_query: - threads.append(thread) - elif skipped >= to_skip: - if len(threads) == per_page: - has_more = True - break - threads.append(thread) - else: - skipped += 1 - num_pages = page + 1 if has_more else page - else: - if raw_query: - threads = list(comment_threads) - else: - page = max(1, page) - paginated_collection = comment_threads.skip( - (page - 1) * per_page - ).limit(per_page) - threads = list(paginated_collection) - num_pages = max(1, math.ceil(thread_count / per_page)) - - if raw_query: - return {"result": threads} - if len(threads) == 0: - collection = [] - else: - thread_ids = [str(thread["_id"]) for thread in threads] - collection = cls.threads_presentor( - thread_ids, user_id, course_id, count_flagged - ) - - return { - "collection": collection, - "num_pages": num_pages, - "page": page, - "thread_count": thread_count, - } - - return {} - - @staticmethod - def prepare_thread( - thread_id: str, - is_read: bool, - unread_count: int, - is_endorsed: bool, - abuse_flagged_count: int, - ) -> dict[str, Any]: - """ - Prepares thread data for presentation. - - Args: - thread (dict[str, Any]): The thread data. - is_read (bool): Whether the thread is read. - unread_count (int): The count of unread comments. - is_endorsed (bool): Whether the thread is endorsed. - abuse_flagged_count (int): The abuse flagged count. - - Returns: - dict[str, Any]: A dictionary representing the prepared thread data. - """ - thread = CommentThread().get(thread_id) - if not thread: - raise ValueError("Thread does not exist.") - - return { - "id": str(thread["_id"]), - **thread, - "type": "thread", - "read": is_read, - "unread_comments_count": unread_count, - "endorsed": is_endorsed, - "abuse_flagged_count": abuse_flagged_count, - } - - @classmethod - def threads_presentor( - cls, - thread_ids: list[str], - user_id: str, - course_id: str, - count_flagged: bool = False, - ) -> list[dict[str, Any]]: - """ - Presents the threads by preparing them for display. - - Args: - threads (list[dict[str, Any]]): List of threads to present. - user_id (str): The ID of the user presenting the threads. - course_id (str): The course ID associated with the threads. - count_flagged (bool, optional): Whether to include flagged content count. Defaults to False. - - Returns: - list[dict[str, Any]]: A list of prepared thread data. - """ - threads = CommentThread().find( - {"_id": {"$in": [ObjectId(thread_id) for thread_id in thread_ids]}} - ) - read_states = cls.get_read_states(thread_ids, user_id, course_id) - threads_endorsed = cls.get_endorsed(thread_ids) - threads_flagged = ( - cls.get_abuse_flagged_count(thread_ids) if count_flagged else {} - ) - threads_dict = {str(thread["_id"]): thread for thread in threads} - - presenters = [] - for thread_id in thread_ids: - thread = threads_dict.get(thread_id) - if thread: - thread_key = thread_id - is_read, unread_count = read_states.get( - thread_key, (False, thread["comment_count"]) - ) - is_endorsed = threads_endorsed.get(thread_key, False) - abuse_flagged_count = threads_flagged.get(thread_key, 0) - presenters.append( - cls.prepare_thread( - thread["_id"], - is_read, - unread_count, - is_endorsed, - abuse_flagged_count, - ) - ) - - return presenters - - @staticmethod - def get_username_from_id(user_id: str) -> Optional[str]: - """ - Retrieve the username associated with a given user ID. - - Args: - _id (int): The unique identifier of the user. - - Returns: - Optional[str]: The username of the user if found, or None if not. - - """ - user = Users().get(_id=user_id) or {} - if username := user.get("username"): - return username - return None - - @staticmethod - def validate_object(model: str, obj_id: str) -> Any: - """ - Validates the object if it exists or not. - - Parameters: - model: The model for which to validate the id. - id: The ID of the object to validate in the model. - Response: - raise exception if object does not exists. - return object - """ - models = { - "Comment": Comment, - "CommentThread": CommentThread, - } - instance = models[model]().get(obj_id) - if not instance: - raise ObjectDoesNotExist - return instance - - @staticmethod - def find_subscribed_threads( - user_id: str, course_id: Optional[str] = None - ) -> list[str]: - """ - Find threads that a user is subscribed to in a specific course. - - Args: - user_id (str): The ID of the user. - course_id (str): The ID of the course. - - Returns: - list: A list of thread ids that the user is subscribed to in the course. - """ - subscriptions = Subscriptions() - threads = CommentThread() - - subscription_filter = {"subscriber_id": user_id} - subscriptions_cursor = subscriptions.find(subscription_filter) - - thread_ids = [] - for subscription in subscriptions_cursor: - thread_ids.append(ObjectId(subscription["source_id"])) - - thread_filter: dict[str, Any] = {"_id": {"$in": thread_ids}} - if course_id: - thread_filter["course_id"] = course_id - threads_cursor = threads.find(thread_filter) - - subscribed_ids = [] - for thread in threads_cursor: - subscribed_ids.append(str(thread["_id"])) - - return subscribed_ids - - @staticmethod - def subscribe_user( - user_id: str, source_id: str, source_type: str - ) -> dict[str, Any] | None: - """Subscribe a user to a source.""" - subscription = Subscriptions().get_subscription(user_id, source_id) - if not subscription: - Subscriptions().insert(user_id, source_id, source_type) - subscription = Subscriptions().get_subscription(user_id, source_id) - if subscription: - subscription["_id"] = str(subscription["_id"]) - return subscription - - @staticmethod - def unsubscribe_user( - user_id: str, source_id: str, source_type: Optional[str] = "" - ) -> None: - """Unsubscribe a user from a source.""" - Subscriptions().delete_subscription(user_id, source_id, source_type=source_type) - - @staticmethod - def delete_comments_of_a_thread(thread_id: str) -> None: - """Delete comments of a thread.""" - for comment in Comment().get_list( - comment_thread_id=ObjectId(thread_id), - depth=0, - parent_id=None, - ): - Comment().delete(comment["_id"]) - - @staticmethod - def delete_subscriptions_of_a_thread(thread_id: str) -> None: - """Delete subscriptions of a thread.""" - for subscription in Subscriptions().get_list( - source_id=thread_id, source_type="CommentThread" - ): - Subscriptions().delete_subscription( - subscription["subscriber_id"], - subscription["source_id"], - source_type="CommentThread", - ) - - @staticmethod - def validate_params(params: dict[str, Any], user_id: Optional[str] = None) -> None: - """ - Validate the request parameters. - Args: - params (dict): The request parameters. - user_id (optional[str]): The Id of the user for validation. - - Returns: - Response: A Response object with an error message if doesn't exist. - """ - valid_params = [ - "course_id", - "author_id", - "thread_type", - "flagged", - "unread", - "unanswered", - "unresponded", - "count_flagged", - "sort_key", - "page", - "per_page", - "request_id", - "commentable_ids", - "group_id", - "group_ids", - "context", - ] - if not user_id: - valid_params.append("user_id") - user_id = params.get("user_id") - - for key in params: - if key not in valid_params: - raise ForumV2RequestError(f"Invalid parameter: {key}") - - if "course_id" not in params: - raise ForumV2RequestError("Missing required parameter: course_id") - - if user_id: - user = Users().get(user_id) - if not user: - raise ForumV2RequestError("User doesn't exist") - - @classmethod - def get_threads( - cls, - params: dict[str, Any], - user_id: str, - serializer: Any, - thread_ids: list[str], - ) -> dict[str, Any]: - """get subscribed or all threads of a specific course for a specific user.""" - count_flagged = bool(params.get("count_flagged", False)) - threads = cls.handle_threads_query( - thread_ids, - user_id, - params["course_id"], - get_group_ids_from_params(params), - params.get("author_id", ""), - params.get("thread_type"), - bool(params.get("flagged", False)), - bool(params.get("unread", False)), - bool(params.get("unanswered", False)), - bool(params.get("unresponded", False)), - count_flagged, - params.get("sort_key", ""), - int(params.get("page", 1)), - int(params.get("per_page", 100)), - commentable_ids=params.get("commentable_ids", []), - is_moderator=params.get("is_moderator", False), - context=params.get("context", "course"), - ) - context: dict[str, Any] = { - "count_flagged": count_flagged, - "include_endorsed": True, - "include_read_state": True, - } - if user_id: - context["user_id"] = user_id - serializer = serializer( - threads.pop("collection"), many=True, context=context, backend=cls - ) - threads["collection"] = serializer.data - return threads - - @staticmethod - def generate_id() -> str: - return str(ObjectId()) - - @staticmethod - def find_or_create_user( - user_id: str, username: Optional[str] = "", default_sort_key: Optional[str] = "" - ) -> str: - """Find or create user.""" - user = Users().get(user_id) - if user: - return user["external_id"] - user_id = Users().insert( - user_id, username=username, default_sort_key=default_sort_key - ) - return user_id - - @classmethod - def create_comment(cls, data: dict[str, Any]) -> str: - """ - handle comment creation and returns a comment. - - Parameters: - data: The content of the comment. - - Response: - The details of the comment that is created. - """ - new_comment_id = Comment().insert( - body=data["body"], - author_id=data["author_id"], - author_username=data.get("author_username"), - course_id=data["course_id"], - anonymous=data.get("anonymous", False), - anonymous_to_peers=data.get("anonymous_to_peers", False), - depth=data.get("depth", 0), - comment_thread_id=data["comment_thread_id"], - parent_id=data.get("parent_id"), - ) - - if data.get("parent_id"): - cls.update_stats_for_course(data["author_id"], data["course_id"], replies=1) - else: - cls.update_stats_for_course( - data["author_id"], data["course_id"], responses=1 - ) - - return str(new_comment_id) - - @staticmethod - def update_comment_and_get_updated_comment( - comment_id: str, - body: Optional[str] = None, - course_id: Optional[str] = None, - user_id: Optional[str] = None, - anonymous: Optional[bool] = False, - anonymous_to_peers: Optional[bool] = False, - endorsed: Optional[bool] = None, - closed: Optional[bool] = False, - editing_user_id: Optional[str] = None, - edit_reason_code: Optional[str] = None, - endorsement_user_id: Optional[str] = None, - ) -> dict[str, Any] | None: - """ - Update an existing child/parent comment. - - Parameters: - comment_id: The ID of the comment to be edited. - body (Optional[str]): The content of the comment. - course_id (Optional[str]): The Id of the respective course. - user_id (Optional[str]): The requesting user id. - anonymous (Optional[bool]): anonymous flag(True or False). - anonymous_to_peers (Optional[bool]): anonymous to peers flag(True or False). - endorsed (Optional[bool]): Flag indicating if the comment is endorsed by any user. - closed (Optional[bool]): Flag indicating if the comment thread is closed. - editing_user_id (Optional[str]): The ID of the user editing the comment. - edit_reason_code (Optional[str]): The reason for editing the comment, typically represented by a code. - endorsement_user_id (Optional[str]): The ID of the user endorsing the comment. - Response: - The details of the comment that is updated. - """ - Comment().update( - comment_id, - body=body, - course_id=course_id, - author_id=user_id, - anonymous=anonymous, - anonymous_to_peers=anonymous_to_peers, - endorsed=endorsed, - closed=closed, - editing_user_id=editing_user_id, - edit_reason_code=edit_reason_code, - endorsement_user_id=endorsement_user_id, - ) - return Comment().get(comment_id) - - @staticmethod - def get_commentables_counts_based_on_type(course_id: str) -> dict[str, Any]: - """Return commentables counts in a course based on thread's type.""" - pipeline: list[dict[str, Any]] = [ - {"$match": {"course_id": course_id, "_type": "CommentThread"}}, - { - "$group": { - "_id": {"topic_id": "$commentable_id", "type": "$thread_type"}, - "count": {"$sum": 1}, - } - }, - ] - - result = CommentThread().aggregate(pipeline) - commentable_counts = {} - for commentable in result: - topic_id = commentable["_id"]["topic_id"] - if topic_id not in commentable_counts: - commentable_counts[topic_id] = {"discussion": 0, "question": 0} - commentable_counts[topic_id].update( - {commentable["_id"]["type"]: commentable["count"]} - ) - - return commentable_counts - - @classmethod - def get_user_voted_ids(cls, user_id: str, vote: str) -> list[str]: - """Get the IDs of the posts voted by a user.""" - if vote not in ["up", "down"]: - raise ValueError("Invalid vote type") - - content_model = Contents() - contents = content_model.get_list() - voted_ids = [] - for content in contents: - votes = content["votes"][vote] - if user_id in votes: - voted_ids.append(content["_id"]) - - return voted_ids - - @staticmethod - def filter_standalone_threads(comment_ids: list[str]) -> list[str]: - """Filter out standalone threads from the list of threads.""" - comments = Comment().find({"_id": {"$in": comment_ids}}) - filtered_comments = [] - for comment in comments: - if not comment["context"] == "standalone": - filtered_comments.append(comment) - return [str([comment["comment_thread_id"]]) for comment in filtered_comments] - - @classmethod - def user_to_hash( - cls, user_id: str, params: Optional[dict[str, Any]] = None - ) -> dict[str, Any]: - """ - Converts user data to a hash - """ - user = Users().get(user_id) - if not user: - raise ValueError("User not found.") - if params is None: - params = {} - - hash_data = {} - hash_data["username"] = user["username"] - hash_data["external_id"] = user["external_id"] - hash_data["id"] = user["external_id"] - - comment_model = Comment() - thread_model = CommentThread() - - if params.get("complete"): - subscribed_thread_ids = cls.find_subscribed_threads(user["external_id"]) - upvoted_ids = cls.get_user_voted_ids(user["external_id"], "up") - downvoted_ids = cls.get_user_voted_ids(user["external_id"], "down") - hash_data.update( - { - "subscribed_thread_ids": subscribed_thread_ids, - "subscribed_commentable_ids": [], - "subscribed_user_ids": [], - "follower_ids": [], - "id": user["external_id"], - "upvoted_ids": upvoted_ids, - "downvoted_ids": downvoted_ids, - "default_sort_key": user["default_sort_key"], - } - ) - - if params.get("course_id"): - threads = thread_model.find( - { - "author_id": user["external_id"], - "course_id": params["course_id"], - "anonymous": False, - "anonymouse_to_peers": False, - } - ) - comments = comment_model.find( - { - "author_id": user["external_id"], - "course_id": params["course_id"], - "anonymous": False, - "anonymouse_to_peers": False, - } - ) - if params.get("group_ids"): - specified_groups_or_global = params["group_ids"] + [None] - group_query = { - "_id": {"$in": [thread["_id"] for thread in threads]}, - "$and": [ - {"group_id": {"$in": specified_groups_or_global}}, - {"group_id": {"$exists": False}}, - ], - } - group_threads = CommentThread().find(group_query) - group_thread_ids = [str(thread["_id"]) for thread in group_threads] - threads_count = len(group_thread_ids) - comment_ids = [comment["_id"] for comment in comments] - comment_thread_ids = cls.filter_standalone_threads(comment_ids) - - group_query = { - "_id": {"$in": [ObjectId(tid) for tid in comment_thread_ids]}, - "$and": [ - {"group_id": {"$in": specified_groups_or_global}}, - {"group_id": {"$exists": False}}, - ], - } - group_comment_threads = thread_model.find(group_query) - group_comment_thread_ids = [ - str(thread["_id"]) for thread in group_comment_threads - ] - comments_count = sum( - 1 - for comment_thread_id in comment_thread_ids - if comment_thread_id in group_comment_thread_ids - ) - else: - thread_ids = [str(thread["_id"]) for thread in threads] - threads_count = len(thread_ids) - comment_ids = [comment["_id"] for comment in comments] - comment_thread_ids = cls.filter_standalone_threads(comment_ids) - comments_count = len(comment_thread_ids) - - hash_data.update( - { - "threads_count": threads_count, - "comments_count": comments_count, - } - ) - - return hash_data - - @staticmethod - def replace_username_in_all_content(user_id: str, username: str) -> None: - """Replace new username in all content documents.""" - content_model = Contents() - contents = content_model.get_list(author_id=user_id) - for content in contents: - content_model.update( - content["_id"], - author_username=username, - ) - - @staticmethod - def unsubscribe_all(user_id: str) -> None: - """Unsubscribe user from all content.""" - subscriptions = Subscriptions() - subscription_filter = {"subscriber_id": user_id} - subscriptions_cursor = subscriptions.find(subscription_filter) - - for subscription in subscriptions_cursor: - subscriptions.delete(subscription["_id"]) - - @staticmethod - def retire_all_content(user_id: str, username: str) -> None: - """Retire all content from user.""" - content_model = Contents() - contents = content_model.get_list(author_id=user_id) - for content in contents: - content_model.update( - content["_id"], - author_username=username, - body=RETIRED_BODY, - ) - if content["_type"] == "CommentThread": - content_model.update( - content["_id"], - title=RETIRED_TITLE, - ) - - @staticmethod - def find_or_create_read_state(user_id: str, thread_id: str) -> dict[str, Any]: - """Find or create user read states.""" - user = Users().get(user_id) - if not user: - raise ObjectDoesNotExist - thread = CommentThread().get(thread_id) - if not thread: - raise ObjectDoesNotExist - - read_states = user.get("read_states", []) - for state in read_states: - if state["course_id"] == thread["course_id"]: - return state - - read_state = { - "_id": ObjectId(), - "course_id": thread["course_id"], - "last_read_times": {}, - } - read_states.append(read_state) - Users().update(user_id, read_states=read_states) - return read_state - - @classmethod - def mark_as_read(cls, user_id: str, thread_id: str) -> None: - """Mark thread as read.""" - user = Users().get(user_id) - thread = CommentThread().get(thread_id) - if not (user and thread): - raise ValueError("User and/or Thread not found.") - read_state = cls.find_or_create_read_state(user["external_id"], thread["_id"]) - - read_state["last_read_times"].update( - { - str(thread["_id"]): datetime.now(timezone.utc), - } - ) - update_user = Users().get(user["external_id"]) - if not update_user: - raise ObjectDoesNotExist - new_read_states = update_user["read_states"] - updated_read_states = [] - for state in new_read_states: - if state["course_id"] == thread["course_id"]: - state = read_state - updated_read_states.append(state) - - Users().update(user["external_id"], read_states=updated_read_states) - - @staticmethod - def find_or_create_user_stats(user_id: str, course_id: str) -> dict[str, Any]: - """Find or create user stats document.""" - user = Users().get(user_id) - if not user: - raise ObjectDoesNotExist - - course_stats = user.get("course_stats", []) - for stat in course_stats: - if stat["course_id"] == course_id: - return stat - - course_stat = { - "_id": ObjectId(), - "active_flags": 0, - "inactive_flags": 0, - "threads": 0, - "responses": 0, - "replies": 0, - "course_id": course_id, - "last_activity_at": "", - } - course_stats.append(course_stat) - Users().update(user["external_id"], course_stats=course_stats) - return course_stat - - @staticmethod - def update_user_stats_for_course(user_id: str, stat: dict[str, Any]) -> None: - """Update user stats for course.""" - user = Users().get(user_id) - if not user: - raise ObjectDoesNotExist - updated_course_stats = [] - course_stats = user["course_stats"] - for course_stat in course_stats: - if course_stat["course_id"] == stat["course_id"]: - course_stat.update(stat) - updated_course_stats.append(course_stat) - Users().update(user_id, course_stats=updated_course_stats) - - @classmethod - def build_course_stats(cls, author_id: str, course_id: str) -> None: - """Build course stats.""" - user = Users().get(author_id) - if not user: - raise ObjectDoesNotExist - pipeline = [ - { - "$match": { - "course_id": course_id, - "author_id": user["external_id"], - "anonymous_to_peers": False, - "anonymous": False, - } - }, - { - "$addFields": { - "is_reply": {"$ne": [{"$ifNull": ["$parent_id", None]}, None]} - } - }, - { - "$group": { - "_id": {"type": "$_type", "is_reply": "$is_reply"}, - "count": {"$sum": 1}, - "active_flags": { - "$sum": { - "$cond": { - "if": {"$gt": [{"$size": "$abuse_flaggers"}, 0]}, - "then": 1, - "else": 0, - } - } - }, - "inactive_flags": { - "$sum": { - "$cond": { - "if": { - "$gt": [{"$size": "$historical_abuse_flaggers"}, 0] - }, - "then": 1, - "else": 0, - } - } - }, - "latest_update_at": {"$max": "$updated_at"}, - } - }, - ] - - data = list(Contents().aggregate(pipeline)) - active_flags = 0 - inactive_flags = 0 - threads = 0 - responses = 0 - replies = 0 - updated_at = datetime.utcfromtimestamp(0) - - for counts in data: - _type, is_reply = counts["_id"]["type"], counts["_id"]["is_reply"] - last_update_at = counts.get("latest_update_at", datetime(1970, 1, 1)) - if _type == "Comment" and is_reply: - replies = counts["count"] - elif _type == "Comment" and not is_reply: - responses = counts["count"] - else: - threads = counts["count"] - last_update_at = make_aware(last_update_at) - updated_at = make_aware(updated_at) - updated_at = max(last_update_at, updated_at) - active_flags += counts["active_flags"] - inactive_flags += counts["inactive_flags"] - - stats = cls.find_or_create_user_stats(user["external_id"], course_id) - stats["replies"] = replies - stats["responses"] = responses - stats["threads"] = threads - stats["active_flags"] = active_flags - stats["inactive_flags"] = inactive_flags - stats["last_activity_at"] = updated_at - cls.update_user_stats_for_course(user["external_id"], stats) - - @classmethod - def update_all_users_in_course(cls, course_id: str) -> list[str]: - """Update all user stats in a course.""" - course_contents = Contents().get_list( - anonymous=False, - anonymous_to_peers=False, - course_id=course_id, - ) - author_ids = [] - for content in course_contents: - if content["author_id"] not in author_ids: - author_ids.append(content["author_id"]) - - for author_id in author_ids: - cls.build_course_stats(author_id, course_id) - return author_ids - - @staticmethod - def get_user_by_username(username: str | None) -> dict[str, Any] | None: - """Return user from username.""" - cursor = Users().find({"username": username}) - try: - return next(cursor) - except StopIteration: - return None - - @staticmethod - def get_comment(comment_id: str) -> dict[str, Any] | None: - """Get comment from id.""" - comment = Comment().get(comment_id) - return comment - - @staticmethod - def get_thread(thread_id: str) -> dict[str, Any] | None: - """Get thread from id.""" - thread = CommentThread().get(thread_id) - if not thread: - return None - return thread - - @staticmethod - def get_comments(**kwargs: Any) -> list[dict[str, Any]]: - """Return comments from kwargs.""" - if "comment_thread_id" in kwargs: - kwargs["comment_thread_id"] = ObjectId(kwargs["comment_thread_id"]) - if parent_id := kwargs.get("parent_id"): - kwargs["parent_id"] = ObjectId(parent_id) - - return list(Comment().get_list(**kwargs)) - - @staticmethod - def get_comments_count(**kwargs: Any) -> int: - """Return comments from kwargs.""" - if "comment_thread_id" in kwargs: - kwargs["comment_thread_id"] = ObjectId(kwargs["comment_thread_id"]) - if parent_id := kwargs.get("parent_id"): - kwargs["parent_id"] = ObjectId(parent_id) - - return Comment().count_documents(kwargs) - - @staticmethod - def update_comment(comment_id: str, **kwargs: Any) -> int: - """Update comment.""" - return Comment().update(comment_id, **kwargs) - - @staticmethod - def delete_comment(comment_id: str) -> None: - """Delete comment.""" - Comment().delete(comment_id) - - @staticmethod - def get_thread_id_from_comment(comment_id: str) -> dict[str, Any] | None: - """Return thread_id from comment_id.""" - parent_comment = Comment().get(comment_id) - if parent_comment: - return parent_comment["comment_thread_id"] - raise ValueError("Comment doesn't have the thread.") - - @staticmethod - def get_user(user_id: str, get_full_dict: bool = True) -> dict[str, Any] | None: - """Return user from user_id.""" - return Users().get(user_id) - - @staticmethod - def get_subscription( - subscriber_id: str, source_id: str, **kwargs: Any - ) -> dict[str, Any] | None: - """Return subscription from subscriber_id and source_id.""" - subscription = Subscriptions().get_subscription(subscriber_id, source_id) - if not subscription: - return None - return subscription - - @staticmethod - def get_subscriptions(query: dict[str, Any]) -> list[dict[str, Any]]: - """Return subscriptions from filter.""" - return list(Subscriptions().find(query)) - - @staticmethod - def delete_thread(thread_id: str) -> int: - """Delete thread.""" - return CommentThread().delete(thread_id) - - @staticmethod - def create_thread(data: dict[str, Any]) -> str: - """Create thread.""" - new_thread_id = CommentThread().insert( - title=data["title"], - body=data["body"], - course_id=data["course_id"], - anonymous=str_to_bool(data.get("anonymous", "False")), - anonymous_to_peers=str_to_bool(data.get("anonymous_to_peers", "False")), - author_id=data["author_id"], - commentable_id=data.get("commentable_id", "course"), - thread_type=data.get("thread_type", "discussion"), - author_username=data.get("author_username"), - context=data.get("context", "course"), - pinned=data.get("pinned", False), - visible=data.get("visible", True), - abuse_flaggers=data.get("abuse_flaggers"), - historical_abuse_flaggers=data.get("historical_abuse_flaggers"), - group_id=data.get("group_id"), - ) - return new_thread_id - - @staticmethod - def update_thread(thread_id: str, **kwargs: Any) -> int: - """Update thread.""" - return CommentThread().update(thread_id, **kwargs) - - @staticmethod - def get_filtered_threads( - query: dict[str, Any], ids_only: bool = False - ) -> list[dict[str, Any]]: - """Return threads from filter.""" - thread_filter = { - "_type": {"$in": [CommentThread().content_type]}, - "course_id": query.get("course_id"), - } - threads = list(CommentThread().find(thread_filter)) - if ids_only: - return [{"_id": thread["_id"]} for thread in threads] - return threads - - @staticmethod - def update_user(user_id: str, data: dict[str, Any]) -> int: - """Update user.""" - return Users().update(user_id, **data) - - @staticmethod - def get_thread_id_by_comment_id(parent_comment_id: str) -> str: - """ - The thread Id from the parent comment. - """ - parent_comment = Comment().get(parent_comment_id) - if parent_comment: - return parent_comment["comment_thread_id"] - raise ValueError("Comment doesn't have the thread.") - - @staticmethod - def get_course_id_by_thread_id(thread_id: str) -> str | None: - """ - Return course_id for the matching thread. - """ - try: - thread = CommentThread().get(thread_id) - if thread: - # As thread can be a standalone thread(without a course_id). - # So, using thread.get() instead of thread[] to avoid key error. - return thread.get("course_id") - except bson_errors.InvalidId: - # this exception occurs when trying to get course_id from thread that exists - # in mysql. Then Id will not be an ObjectID. So bypassing this exception will - # let it search in mysql. - pass - return None - - @staticmethod - def get_course_id_by_comment_id(comment_id: str) -> str | None: - """ - Return course_id for the matching comment. - """ - try: - comment = Comment().get(comment_id) - if comment: - # As comment can be a standalone comment(comment on a standalone thread). - # So, using comment.get() instead of comment[] to avoid key error. - return comment.get("course_id") - except bson_errors.InvalidId: - # this exception occurs when trying to get course_id from comment that exists - # in mysql. Then Id will not be an ObjectID. So bypassing this exception will - # let it search in mysql. - pass - return None - - @staticmethod - def get_users(**kwargs: Any) -> list[dict[str, Any]]: - """Get users.""" - return list(Users().get_list(**kwargs)) - - @staticmethod - def get_user_sort_criterion(sort_by: str) -> dict[str, Any]: - """Get sort criterion based on sort_by parameter.""" - if sort_by == "flagged": - return { - "course_stats.active_flags": -1, - "course_stats.inactive_flags": -1, - "username": -1, - } - elif sort_by == "recency": - return { - "course_stats.last_activity_at": -1, - "username": -1, - } - else: - return { - "course_stats.threads": -1, - "course_stats.responses": -1, - "course_stats.replies": -1, - "username": -1, - } - - @staticmethod - def create_user_pipeline( - course_id: str, page: int, per_page: int, sort_criterion: dict[str, Any] - ) -> list[dict[str, Any]]: - """Get pipeline for course stats api.""" - pipeline: list[dict[str, Any]] = [ - {"$match": {"course_stats.course_id": course_id}}, - {"$project": {"username": 1, "course_stats": 1}}, - {"$unwind": "$course_stats"}, - {"$match": {"course_stats.course_id": course_id}}, - {"$sort": sort_criterion}, - { - "$facet": { - "pagination": [{"$count": "total_count"}], - "data": [ - {"$skip": (page - 1) * per_page}, - {"$limit": per_page}, - ], - } - }, - ] - return pipeline - - # pylint: disable=E1121 - @classmethod - def get_paginated_user_stats( - cls, course_id: str, page: int, per_page: int, sort_criterion: dict[str, Any] - ) -> dict[str, Any]: - """Get paginated stats for a course.""" - pipeline = cls.create_user_pipeline(course_id, page, per_page, sort_criterion) - return list(Users().aggregate(pipeline))[0] - - @staticmethod - def get_contents(**kwargs: Any) -> list[dict[str, Any]]: - """Return contents.""" - return list(Contents().get_list(**kwargs)) - - @staticmethod - def get_user_thread_filter(course_id: str) -> dict[str, Any]: - """Get user thread filter.""" - return { - "_type": {"$in": [CommentThread.content_type]}, - "course_id": {"$in": [course_id]}, - } - - @staticmethod - def find_thread(**kwargs: Any) -> Optional[dict[str, Any]]: - """ - Retrieves a first matching thread from the database. - """ - return CommentThread().find_one(kwargs) - - @staticmethod - def find_comment( - is_parent_comment: bool = True, with_abuse_flaggers: bool = False, **kwargs: Any - ) -> Optional[dict[str, Any]]: - """ - Retrieves a first matching comment from the database. - """ - if is_parent_comment: - kwargs["parent_id"] = None - else: - kwargs["parent_id"] = {"$ne": None} - if with_abuse_flaggers: - kwargs["abuse_flaggers"] = {"$ne": []} - - return Comment().find_one(kwargs) - - @staticmethod - def get_user_contents_by_username(username: str) -> list[dict[str, Any]]: - """ - Retrieve all threads and comments authored by a specific user. - """ - contents = list(Comment().find({"author_username": username})) + list( - CommentThread().find({"author_username": username}) - ) - return contents - - @staticmethod - def get_user_post_counts(user_id: str, course_id: str) -> dict[str, int]: - """Return thread_count and comment_count for user in course.""" - query = {"author_id": user_id, "course_id": course_id} - thread_count = CommentThread().count_documents( - {**query, "_type": "CommentThread"} - ) - comment_count = Comment().count_documents({**query, "_type": "Comment"}) - return {"thread_count": thread_count, "comment_count": comment_count} - - @staticmethod - def delete_user_posts(user_id: str, course_id: str) -> dict[str, int]: - """Delete all threads and comments by user in course. Returns counts before deletion.""" - thread_model = CommentThread() - comment_model = Comment() - query = {"author_id": user_id, "course_id": course_id} - thread_count = thread_model.count_documents({**query, "_type": "CommentThread"}) - comment_count = comment_model.count_documents({**query, "_type": "Comment"}) - # Collect IDs before deleting to avoid cursor invalidation. - # find() uses override_query which adds _type automatically. - comment_ids = [str(c["_id"]) for c in comment_model.find(query)] - for comment_id in comment_ids: - comment_model.delete(comment_id) - thread_ids = [str(t["_id"]) for t in thread_model.find(query)] - for thread_id in thread_ids: - thread_model.delete(thread_id) - return {"thread_count": thread_count, "comment_count": comment_count} diff --git a/forum/backends/mongodb/base_model.py b/forum/backends/mongodb/base_model.py deleted file mode 100644 index fbc770ad..00000000 --- a/forum/backends/mongodb/base_model.py +++ /dev/null @@ -1,127 +0,0 @@ -""" -Database models for forum. -""" - -from abc import ABC -from typing import Any, Optional - -from bson import ObjectId -from pymongo.collection import Collection as PymongoCollection -from pymongo.command_cursor import CommandCursor -from pymongo.cursor import Cursor - -from forum.mongo import Database, get_database - -Collection = PymongoCollection[dict[str, Any]] - - -class MongoBaseModel(ABC): - """Abstract Class for Mongo model implementation""" - - MONGODB_DATABASE: Optional[Database] = None - COLLECTION_NAME: str = "default" - index_name: str = "default" - - @property - def _collection(self) -> Collection: - """Return the MongoDB collection for the model.""" - return self.__get_database()[self.COLLECTION_NAME] - - @classmethod - def __get_database(cls) -> Database: - """Get or create static class database.""" - if cls.MONGODB_DATABASE is None: - cls.MONGODB_DATABASE = get_database() - return cls.MONGODB_DATABASE - - def override_query(self, query: dict[str, Any]) -> dict[str, Any]: - """Override Query""" - return query - - def get(self, _id: str) -> Optional[dict[str, Any]]: - """Get a document by ID.""" - return self._collection.find_one({"_id": ObjectId(_id)}) - - def get_list(self, **kwargs: Any) -> Cursor[dict[str, Any]]: - """Get a list of all documents filtered by kwargs.""" - return self._collection.find(kwargs) - - def delete(self, _id: str) -> int: - """ - Delete a document from the database based on the ID. - - Args: - _id: The ID of the document. - - Returns: - The number of documents deleted. - """ - result = self._collection.delete_one({"_id": ObjectId(_id)}) - return result.deleted_count - - def find(self, query: dict[str, Any]) -> Cursor[dict[str, Any]]: - """ - Run a raw MongoDB query. - - Args: - query: The MongoDB query. - - Returns: - A cursor with the query results. - """ - query = self.override_query(query) - return self._collection.find(query) - - def find_one(self, query: dict[str, Any]) -> Optional[dict[str, Any]]: - """ - Run a raw MongoDB query to find a single document. - - Args: - query: The MongoDB query. - - Returns: - The first document matching the query, or None if no document matches. - """ - query = self.override_query(query) - return self._collection.find_one(query) - - def aggregate( - self, pipeline: list[dict[str, Any]] - ) -> CommandCursor[dict[str, Any]]: - """ - Run a MongoDB aggregation pipeline. - - Args: - pipeline: The aggregation pipeline. - - Returns: - A command cursor with the aggregation results. - """ - return self._collection.aggregate(pipeline) - - def count_documents(self, query: dict[str, Any]) -> int: - """ - Count the number of documents matching a query. - - Args: - query: The MongoDB query. - - Returns: - The count of documents matching the query. - """ - query = self.override_query(query) - return self._collection.count_documents(query) - - def distinct(self, field: str, query: dict[str, Any]) -> list[Any]: - """ - Run a MongoDB distinct query. - - Args: - field: The field for which to return distinct values. - query: The MongoDB query. - - Returns: - A list of distinct values for the specified field. - """ - query = self.override_query(query) - return self._collection.distinct(field, query) diff --git a/forum/backends/mongodb/comments.py b/forum/backends/mongodb/comments.py deleted file mode 100644 index a50563c2..00000000 --- a/forum/backends/mongodb/comments.py +++ /dev/null @@ -1,365 +0,0 @@ -"""Comment Class for mongo backend.""" - -from datetime import datetime -from typing import Any, Optional - -from bson import ObjectId - -from forum.backends.mongodb.contents import BaseContents -from forum.backends.mongodb.threads import CommentThread -from forum.backends.mongodb.users import Users -from forum.utils import get_handler_by_name - - -class Comment(BaseContents): - """ - Comment class for cs_comments_service content model - """ - - index_name = "comments" - content_type = "Comment" - - def override_query(self, query: dict[str, Any]) -> dict[str, Any]: - query = {**query, "_type": self.content_type} - return super().override_query(query) - - @classmethod - def mapping(cls) -> dict[str, Any]: - """ - Mapping function for the Thread class - """ - return { - "dynamic": "false", - "properties": { - "body": { - "type": "text", - "store": True, - "term_vector": "with_positions_offsets", - }, - "course_id": {"type": "keyword"}, - "comment_thread_id": {"type": "keyword"}, - "commentable_id": {"type": "keyword"}, - "group_id": {"type": "keyword"}, - "context": {"type": "keyword"}, - "created_at": {"type": "date"}, - "updated_at": {"type": "date"}, - "title": {"type": "keyword"}, - }, - } - - @classmethod - def doc_to_hash(cls, doc: dict[str, Any]) -> dict[str, Any]: - """ - Converts comment document to the dict - """ - return { - "body": doc.get("body"), - "course_id": doc.get("course_id"), - "comment_thread_id": str(doc.get("comment_thread_id")), - "commentable_id": doc.get("commentable_id"), - "group_id": doc.get("group_id"), - "context": doc.get("context", "course"), - "created_at": doc.get("created_at"), - "updated_at": doc.get("updated_at"), - "title": doc.get("title"), - } - - def insert( - self, - body: str, - course_id: str, - author_id: str, - comment_thread_id: str, - parent_id: Optional[str] = None, - author_username: Optional[str] = None, - anonymous: bool = False, - anonymous_to_peers: bool = False, - depth: int = 0, - abuse_flaggers: Optional[list[str]] = None, - historical_abuse_flaggers: Optional[list[str]] = None, - visible: bool = True, - ) -> str: - """ - Inserts a new comment document into the database. - - Args: - body (str): The body content of the comment. - course_id (str): The ID of the course the comment is associated with. - comment_thread_id (str): The ID of the parent comment thread. - author_id (str): The ID of the author who created the comment. - author_username (str): The username of the author. - anonymous (bool, optional): Whether the comment is posted anonymously. Defaults to False. - anonymous_to_peers (bool, optional): Whether the comment is anonymous to peers. Defaults to False. - depth (int, optional): The depth of the comment in the thread hierarchy. Defaults to 0. - abuse_flaggers (Optional[list[str]], optional): Users who flagged the comment. Defaults to None. - historical_abuse_flaggers (Optional[list[str]], optional): Users historically flagged the comment. - visible (bool, optional): Whether the comment is visible. Defaults to True. - - Returns: - str: The ID of the inserted document. - """ - date = datetime.now() - comment_data = { - "votes": self.get_votes_dict(up=[], down=[]), - "visible": visible, - "abuse_flaggers": abuse_flaggers or [], - "historical_abuse_flaggers": historical_abuse_flaggers or [], - "parent_ids": [ObjectId(parent_id)] if parent_id else [], - "at_position_list": [], - "body": body, - "course_id": course_id, - "_type": self.content_type, - "endorsed": False, - "anonymous": anonymous, - "anonymous_to_peers": anonymous_to_peers, - "author_id": author_id, - "comment_thread_id": ObjectId(comment_thread_id), - "child_count": 0, - "depth": depth, - "author_username": author_username or self.get_author_username(author_id), - "created_at": date, - "updated_at": date, - } - if parent_id: - comment_data["parent_id"] = ObjectId(parent_id) - - comment_data["endorsement"] = None - - result = self._collection.insert_one(comment_data) - - if parent_id: - self.update_child_count_in_parent_comment(parent_id, 1) - - self.update_comment_count_in_comment_thread(comment_thread_id, 1) - - # Notify Comment inserted - get_handler_by_name("comment_inserted").send( - sender=self.__class__, comment_id=str(result.inserted_id) - ) - - self.update_sk(str(result.inserted_id), parent_id) - return str(result.inserted_id) - - def update( - self, - comment_id: str, - body: Optional[str] = None, - course_id: Optional[str] = None, - anonymous: Optional[bool] = None, - anonymous_to_peers: Optional[bool] = None, - comment_thread_id: Optional[ObjectId] = None, - at_position_list: Optional[list[str]] = None, - visible: Optional[bool] = None, - author_id: Optional[str] = None, - author_username: Optional[str] = None, - votes: Optional[dict[str, int]] = None, - abuse_flaggers: Optional[list[str]] = None, - historical_abuse_flaggers: Optional[list[str]] = None, - endorsed: Optional[bool] = None, - child_count: Optional[int] = None, - depth: Optional[int] = None, - closed: Optional[bool] = None, - editing_user_id: Optional[str] = None, - edit_reason_code: Optional[str] = None, - endorsement_user_id: Optional[str] = None, - sk: Optional[str] = None, - ) -> int: - """ - Updates a comment document in the database. - - Args: - comment_id (ObjectId): The ID of the comment to update. - body (Optional[str], optional): The body content of the comment. - course_id (Optional[str], optional): The ID of the course the comment is associated with. - anonymous (Optional[bool], optional): Whether the comment is posted anonymously. - anonymous_to_peers (Optional[bool], optional): Whether the comment is anonymous to peers. - comment_thread_id (Optional[ObjectId], optional): The ID of the parent comment thread. - at_position_list (Optional[list[str]], optional): A list of positions for @mentions. - visible (Optional[bool], optional): Whether the comment is visible. - author_id (Optional[str], optional): The ID of the author who created the comment. - author_username (Optional[str], optional): The username of the author. - votes (Optional[dict[str, int]], optional): The votes for the comment. - abuse_flaggers (Optional[list[str]], optional): A list of users who flagged the comment for abuse. - historical_abuse_flaggers (Optional[list[str]], optional): Users who historically flagged the comment. - endorsed (Optional[bool], optional): Whether the comment is endorsed. - child_count (Optional[int], optional): The number of child comments. - depth (Optional[int], optional): The depth of the comment in the thread hierarchy. - - Returns: - int: The number of documents modified. - """ - fields = [ - ("body", body), - ("course_id", course_id), - ("anonymous", anonymous), - ("anonymous_to_peers", anonymous_to_peers), - ("comment_thread_id", comment_thread_id), - ("at_position_list", at_position_list), - ("visible", visible), - ("author_id", author_id), - ("author_username", author_username), - ("votes", votes), - ("abuse_flaggers", abuse_flaggers), - ("historical_abuse_flaggers", historical_abuse_flaggers), - ("endorsed", endorsed), - ("child_count", child_count), - ("depth", depth), - ("closed", closed), - ("sk", sk), - ] - update_data: dict[str, Any] = { - field: value for field, value in fields if value is not None - } - if endorsed and endorsement_user_id: - update_data["endorsement"] = { - "user_id": endorsement_user_id, - "time": datetime.now(), - } - else: - update_data["endorsement"] = None - - if editing_user_id: - edit_history = [] - original_body = "" - if comment := Comment().get(comment_id): - edit_history = comment.get("edit_history", []) - original_body = comment.get("body", "") - edit_history.append( - { - "author_id": editing_user_id, - "original_body": original_body, - "reason_code": edit_reason_code, - "editor_username": self.get_author_username(editing_user_id), - "created_at": datetime.now(), - } - ) - update_data["edit_history"] = edit_history - - update_data["updated_at"] = datetime.now() - result = self._collection.update_one( - {"_id": ObjectId(comment_id)}, - {"$set": update_data}, - ) - - # Notify Comment updated - get_handler_by_name("comment_updated").send( - sender=self.__class__, comment_id=comment_id - ) - - return result.modified_count - - def delete(self, _id: str) -> int: - """ - Deletes a comment from the database based on the id. - - Args: - _id: The ID of the comment. - - Returns: - The number of comments deleted. - """ - comment = self.get(_id) - if not comment: - return 0 - - parent_comment_id = comment.get("parent_id") - child_comments_deleted_count = 0 - if not parent_comment_id: - child_comments_deleted_count = self.delete_child_comments(_id) - - result = self._collection.delete_one({"_id": ObjectId(_id)}) - if parent_comment_id: - self.update_child_count_in_parent_comment(parent_comment_id, -1) - - no_of_comments_delete = result.deleted_count + child_comments_deleted_count - comment_thread_id = comment["comment_thread_id"] - - self.update_comment_count_in_comment_thread( - comment_thread_id, -(int(no_of_comments_delete)) - ) - - # Notify Comments deleted - get_handler_by_name("comment_deleted").send( - sender=self.__class__, comment_id=_id - ) - - return no_of_comments_delete - - def get_author_username(self, author_id: str) -> str | None: - """Return username for the respective author_id(user_id)""" - user = Users().get(author_id) - return user.get("username") if user else None - - def delete_child_comments(self, _id: str) -> int: - """ - Delete child comments from the database based on the id. - - Args: - _id: The ID of the parent comment whose child comments will be deleted. - - Returns: - The number of child comments deleted. - """ - child_comments_to_delete = self.find({"parent_id": ObjectId(_id)}) - child_comment_ids_to_delete = [ - child_comment.get("_id") for child_comment in child_comments_to_delete - ] - child_comments_deleted = self._collection.delete_many( - {"_id": {"$in": child_comment_ids_to_delete}} - ) - - for child_comment_id in child_comment_ids_to_delete: - get_handler_by_name("comment_deleted").send( - sender=self.__class__, comment_id=child_comment_id - ) - - return child_comments_deleted.deleted_count - - def update_child_count_in_parent_comment(self, parent_id: str, count: int) -> None: - """ - Update(increment/decrement) child_count in parent comment. - - Args: - parent_id: The ID of the parent comment whose child_count will be updated. - count: It can be any number. - If positive, this function will increase child_count by the count. - If negative, this function will decrease child_count by the count. - - Returns: - None. - """ - update_child_count_query = {"$inc": {"child_count": count}} - self.update_count(parent_id, update_child_count_query) - - def update_comment_count_in_comment_thread( - self, comment_thread_id: str, count: int - ) -> None: - """ - Update(increment/decrement) comment_count in comment thread. - - Args: - comment_thread_id: The ID of the comment thread - whose comment_count will be updated. - count: It can be any number. - If positive, this function will increase comment_count by the count. - If negative, this function will decrease comment_count by the count. - - Returns: - None. - """ - update_comment_count_query = { - "$inc": {"comment_count": count}, - "$set": {"last_activity_at": datetime.now()}, - } - CommentThread().update_count(comment_thread_id, update_comment_count_query) - - def get_sk(self, _id: str, parent_id: Optional[str]) -> str: - """Returns sk field.""" - if parent_id is not None: - return f"{parent_id}-{_id}" - return f"{_id}" - - def update_sk(self, _id: str, parent_id: Optional[str]) -> None: - """Updates sk field.""" - sk = self.get_sk(_id, parent_id) - self.update(_id, sk=sk) diff --git a/forum/backends/mongodb/contents.py b/forum/backends/mongodb/contents.py deleted file mode 100644 index 489ccd2a..00000000 --- a/forum/backends/mongodb/contents.py +++ /dev/null @@ -1,268 +0,0 @@ -"""Content Class for mongo backend.""" - -from datetime import datetime -from typing import Any, Optional - -from bson import ObjectId -from pymongo.cursor import Cursor - -from forum.backends.mongodb.base_model import MongoBaseModel - - -class BaseContents(MongoBaseModel): - """ - BaseContents class: same as Contents, but without "update" and "insert" methods, - because child classes will have different signatures for these methods. - """ - - content_type: str = "" - COLLECTION_NAME: str = "contents" - - def create_indexes(self) -> None: - """ - The implementation creates the indexes in the mongodb for the contents collection. - """ - self._collection.create_index( - [ - ("_type", 1), - ("course_id", 1), - ("pinned", -1), - ("created_at", -1), - ], - background=True, - ) - self._collection.create_index( - [ - ("_type", 1), - ("course_id", 1), - ("pinned", -1), - ("comment_count", -1), - ("created_at", -1), - ], - background=True, - ) - self._collection.create_index( - [ - ("_type", 1), - ("course_id", 1), - ("pinned", -1), - ("votes.point", -1), - ("created_at", -1), - ], - background=True, - ) - self._collection.create_index( - [ - ("_type", 1), - ("course_id", 1), - ("pinned", -1), - ("last_activity_at", -1), - ("created_at", -1), - ], - background=True, - ) - self._collection.create_index( - [ - ("comment_thread_id", 1), - ("sk", 1), - ], - sparse=True, - ) - self._collection.create_index( - [ - ("comment_thread_id", 1), - ("endorsed", 1), - ], - sparse=True, - ) - self._collection.create_index( - [ - ("commentable_id", 1), - ], - sparse=True, - background=True, - ) - - @classmethod - def mapping(cls) -> dict[str, Any]: - """ - Implement this method in the child class - """ - raise NotImplementedError - - @classmethod - def doc_to_hash(cls, doc: dict[str, Any]) -> dict[str, Any]: - """ - Implement this method in the child class - """ - raise NotImplementedError - - def override_query(self, query: dict[str, Any]) -> dict[str, Any]: - """ - Override the query with the _type field. - """ - if self.content_type: - query = {**query, "_type": self.content_type} - return super().override_query(query) - - def get_list(self, **kwargs: Any) -> Cursor[dict[str, Any]]: - """ - Retrieves a list of all content documents in the database based on provided filters. - - Args: - kwargs: The filter arguments. - - Returns: - A list of content documents. - """ - resp_skip = kwargs.pop("resp_skip", 0) - resp_limit = kwargs.pop("resp_limit", None) - - if self.content_type: - kwargs["_type"] = self.content_type - - # Apply sorting first if provided - sort = kwargs.pop("sort", None) - query = self._collection.find(kwargs) - if sort: - query = query.sort("sk", sort) - - # Apply pagination after sorting - query = query.skip(resp_skip) - if resp_limit is not None: - query = query.limit(resp_limit) - - return query - - @classmethod - def get_votes_dict(cls, up: list[str], down: list[str]) -> dict[str, Any]: - """ - Calculates and returns the vote summary for a thread. - - Args: - up (list): A list of user IDs who upvoted the thread. - down (list): A list of user IDs who downvoted the thread. - - Returns: - dict: A dictionary containing the vote summary with the following keys: - - "up" (list): The list of user IDs who upvoted. - - "down" (list): The list of user IDs who downvoted. - - "up_count" (int): The count of upvotes. - - "down_count" (int): The count of downvotes. - - "count" (int): The total number of votes (upvotes + downvotes). - - "point" (int): The vote score (upvotes - downvotes). - """ - up = up or [] - down = down or [] - votes = { - "up": up, - "down": down, - "up_count": len(up), - "down_count": len(down), - "count": len(up) + len(down), - "point": len(up) - len(down), - } - return votes - - def update_votes(self, content_id: str, votes: dict[str, Any]) -> int: - """ - Updates a votes in the content document. - - Args: - content_id: The id of the content model - votes (Optional[dict[str, int]], optional): The votes for the thread. - """ - update_data = {"votes": votes, "updated_at": datetime.now()} - result = self._collection.update_one( - {"_id": ObjectId(content_id)}, - {"$set": update_data}, - ) - return result.modified_count - - def update_count(self, content_id: str, query: dict[str, Any]) -> int: - """ - Updates count of a field in the content document based on query. - - Args: - content_id (str): The id of the content(Commentthread id or Comment id) model. - query (dict[str, Any]): Query to update the count in a specific field. - """ - result = self._collection.update_one( - {"_id": ObjectId(content_id)}, - query, - ) - return result.modified_count - - -class Contents(BaseContents): - """ - Contents class for cs_comments_service contents collection - """ - - def insert( - self, - _id: str, - author_id: str, - abuse_flaggers: list[str], - historical_abuse_flaggers: list[str], - visible: bool, - ) -> str: - """ - Inserts a new content document into the database. - - Args: - _id (str): The ID of the content. - author_id (str): The ID of the author who created the content. - abuse_flaggers (list[str]): A list of IDs of users who flagged the content as abusive. - historical_abuse_flaggers (list[str]): A list of IDs of users who previously flagged the content as abusive. - visible (bool): Whether the content is visible or not. - - Returns: - str: The ID of the inserted document. - """ - content_data = { - "_id": ObjectId(_id), - "author_id": author_id, - "abuse_flaggers": abuse_flaggers, - "historical_abuse_flaggers": historical_abuse_flaggers, - "visible": visible, - } - result = self._collection.insert_one(content_data) - return str(result.inserted_id) - - def update( - self, - _id: str, - author_username: Optional[str] = None, - abuse_flaggers: Optional[list[str]] = None, - historical_abuse_flaggers: Optional[list[str]] = None, - body: Optional[str] = None, - title: Optional[str] = None, - **kwargs: Any - ) -> int: - """ - Updates a contents document in the database based on the provided _id. - - Args: - _id: The id of the contents document to update. - **kwargs: The fields to update in the contents document. - - Returns: - The number of documents modified. - """ - fields = [ - ("author_username", author_username), - ("abuse_flaggers", abuse_flaggers), - ("historical_abuse_flaggers", historical_abuse_flaggers), - ("body", body), - ("title", title), - ] - update_data: dict[str, Any] = { - field: value for field, value in fields if value is not None - } - - result = self._collection.update_one( - {"_id": ObjectId(_id)}, - {"$set": update_data}, - ) - return result.modified_count diff --git a/forum/backends/mongodb/subscriptions.py b/forum/backends/mongodb/subscriptions.py deleted file mode 100644 index 99b36743..00000000 --- a/forum/backends/mongodb/subscriptions.py +++ /dev/null @@ -1,99 +0,0 @@ -"""Subscriptions class for mongo backend.""" - -from datetime import datetime, timezone -from typing import Any, Optional - -from forum.backends.mongodb.base_model import MongoBaseModel - - -class Subscriptions(MongoBaseModel): - """ - Represents a subscription to a source in the MongoDB database. - - This class provides methods for inserting, updating, retrieving, and listing subscriptions. - """ - - COLLECTION_NAME: str = "subscriptions" - - def insert(self, subscriber_id: str, source_id: str, source_type: str) -> str: - """ - Inserts a new subscription into the MongoDB collection. - - Args: - subscriber_id: The ID of the subscriber. - source_id: The ID of the source. - source_type: The type of the source. - - """ - subscription = { - "subscriber_id": subscriber_id, - "source_id": source_id, - "source_type": source_type, - "created_at": datetime.now(timezone.utc), - "updated_at": datetime.now(timezone.utc), - } - result = self._collection.insert_one(subscription) - return str(result.inserted_id) - - def update(self, subscriber_id: str, source_id: str, **kwargs: Any) -> int: - """ - Updates an existing subscription in the MongoDB collection. - - Args: - subscriber_id: The ID of the subscriber. - source_id: The ID of the source. - **kwargs: Additional fields to update. - - """ - filter_query = { - "subscriber_id": subscriber_id, - "source_id": source_id, - } - update_query = {"$set": kwargs, "$currentDate": {"updated_at": True}} - result = self._collection.update_one(filter_query, update_query) - return result.modified_count - - def get_subscription( - self, subscriber_id: str, source_id: str - ) -> Optional[dict[str, Any]]: - """ - Retrieves a subscription from the MongoDB collection. - - Args: - subscriber_id: The ID of the subscriber. - source_id: The ID of the source. - - Returns: - The subscription document if found, otherwise None. - - """ - filter_query = { - "subscriber_id": subscriber_id, - "source_id": source_id, - } - subscription = self._collection.find_one(filter_query) - return subscription - - def delete_subscription( - self, subscriber_id: str, source_id: str, source_type: Optional[str] = "" - ) -> int: - """ - Deletes a subscription from the MongoDB collection. - - Args: - subscriber_id: The ID of the subscriber. - source_id: The ID of the source. - - Returns: - The number of deleted documents. - - """ - filter_query = { - "subscriber_id": subscriber_id, - "source_id": source_id, - } - if source_type: - filter_query["source_type"] = source_type - - result = self._collection.delete_one(filter_query) - return result.deleted_count diff --git a/forum/backends/mongodb/threads.py b/forum/backends/mongodb/threads.py deleted file mode 100644 index be8e9638..00000000 --- a/forum/backends/mongodb/threads.py +++ /dev/null @@ -1,299 +0,0 @@ -"""Content Class for mongo backend.""" - -from datetime import datetime -from typing import Any, Optional - -from bson import ObjectId - -from forum.backends.mongodb.contents import BaseContents -from forum.backends.mongodb.users import Users -from forum.utils import get_handler_by_name - - -class CommentThread(BaseContents): - """ - CommentThread class for cs_comments_service content model - """ - - index_name = "comment_threads" - content_type = "CommentThread" - - def delete(self, _id: str) -> int: - """Delete CommentThread""" - result = super().delete(_id) - get_handler_by_name("comment_thread_deleted").send( - sender=self.__class__, comment_thread_id=_id - ) - Users().delete_read_state_by_thread_id(_id) - return result - - @classmethod - def mapping(cls) -> dict[str, Any]: - """ - Mapping function for the Thread class - """ - return { - "dynamic": "false", - "properties": { - "title": { - "type": "text", - "boost": 5.0, - "store": True, - "term_vector": "with_positions_offsets", - }, - "body": { - "type": "text", - "store": True, - "term_vector": "with_positions_offsets", - }, - "created_at": {"type": "date"}, - "updated_at": {"type": "date"}, - "last_activity_at": {"type": "date"}, - "comment_count": {"type": "integer"}, - "votes_point": {"type": "integer"}, - "context": {"type": "keyword"}, - "course_id": {"type": "keyword"}, - "commentable_id": {"type": "keyword"}, - "author_id": {"type": "keyword"}, - "group_id": {"type": "integer"}, - "id": {"type": "keyword"}, - "thread_id": {"type": "keyword"}, - }, - } - - @classmethod - def doc_to_hash(cls, doc: dict[str, Any]) -> dict[str, Any]: - """ - Converts thread document to the dict - """ - return { - "id": str(doc.get("_id")), - "title": doc.get("title"), - "body": doc.get("body"), - "created_at": doc.get("created_at"), - "updated_at": doc.get("updated_at"), - "last_activity_at": doc.get("last_activity_at"), - "comment_count": doc.get("comment_count"), - "votes_point": doc.get("votes", {}).get("point"), - "context": doc.get("context"), - "course_id": doc.get("course_id"), - "commentable_id": doc.get("commentable_id"), - "author_id": doc.get("author_id"), - "group_id": doc.get("group_id"), - "thread_id": str(doc.get("_id")), - } - - def insert( - self, - title: str, - body: str, - course_id: str, - commentable_id: str, - author_id: str, - author_username: Optional[str] = None, - anonymous: bool = False, - anonymous_to_peers: bool = False, - thread_type: str = "discussion", - context: str = "course", - pinned: bool = False, - visible: bool = True, - abuse_flaggers: Optional[list[str]] = None, - historical_abuse_flaggers: Optional[list[str]] = None, - group_id: Optional[int] = None, - ) -> str: - """ - Inserts a new thread document into the database. - - Args: - title (str): The title of the thread. - body (str): The body content of the thread. - course_id (str): The ID of the course the thread is associated with. - commentable_id (str): The ID of the commentable entity. - author_id (str): The ID of the author who created the thread. - author_username (str): The username of the author. - anonymous (bool, optional): Whether the thread is posted anonymously. Defaults to False. - anonymous_to_peers (bool, optional): Whether the thread is anonymous to peers. Defaults to False. - thread_type (str, optional): The type of the thread, either 'question' or 'discussion'. - context (str, optional): The context of the thread, either 'course' or 'standalone'. - pinned (bool): Whether the thread is pinned. Defaults to False. - visible (bool): Whether the thread is visible. Defaults to True. - abuse_flaggers: A list of users who flagged the thread for abuse. - historical_abuse_flaggers: A list of users who historically flagged the thread for abuse. - - Raises: - ValueError: If `thread_type` is not 'question' or 'discussion'. - ValueError: If `context` is not 'course' or 'standalone'. - - Returns: - str: The ID of the inserted document. - """ - if thread_type not in ["question", "discussion"]: - raise ValueError("Invalid thread_type") - - if context not in ["course", "standalone"]: - raise ValueError("Invalid context") - - if abuse_flaggers is None: - abuse_flaggers = [] - if historical_abuse_flaggers is None: - historical_abuse_flaggers = [] - - date = datetime.now() - thread_data = { - "votes": self.get_votes_dict(up=[], down=[]), - "thread_type": thread_type, - "context": context, - "comment_count": 0, - "at_position_list": [], - "title": title, - "body": body, - "course_id": course_id, - "commentable_id": commentable_id, - "_type": self.content_type, - "anonymous": anonymous, - "anonymous_to_peers": anonymous_to_peers, - "closed": False, - "author_id": author_id, - "author_username": author_username or self.get_author_username(author_id), - "created_at": date, - "updated_at": date, - "last_activity_at": date, - "pinned": pinned, - "visible": visible, - "abuse_flaggers": abuse_flaggers, - "historical_abuse_flaggers": historical_abuse_flaggers, - } - if group_id: - thread_data["group_id"] = group_id - - result = self._collection.insert_one(thread_data) - thread_id = str(result.inserted_id) - - # Notify Thread inserted - get_handler_by_name("comment_thread_inserted").send( - sender=self.__class__, comment_thread_id=thread_id - ) - return thread_id - - def update( - self, - thread_id: str, - thread_type: Optional[str] = None, - title: Optional[str] = None, - body: Optional[str] = None, - course_id: Optional[str] = None, - anonymous: Optional[bool] = None, - anonymous_to_peers: Optional[bool] = None, - commentable_id: Optional[str] = None, - at_position_list: Optional[list[str]] = None, - closed: Optional[bool] = None, - context: Optional[str] = None, - author_id: Optional[str] = None, - author_username: Optional[str] = None, - votes: Optional[dict[str, int]] = None, - abuse_flaggers: Optional[list[str]] = None, - historical_abuse_flaggers: Optional[list[str]] = None, - closed_by: Optional[str] = None, - pinned: Optional[bool] = None, - comments_count: Optional[int] = None, - endorsed: Optional[bool] = None, - edit_history: Optional[list[dict[str, Any]]] = None, - original_body: Optional[str] = None, - editing_user_id: Optional[str] = None, - edit_reason_code: Optional[str] = None, - close_reason_code: Optional[str] = None, - closed_by_id: Optional[str] = None, - group_id: Optional[int] = None, - skip_timestamp_update: bool = False, - ) -> int: - """ - Updates a thread document in the database. - - Args: - thread_id: ID of thread to update. - thread_type: The type of the thread, either 'question' or 'discussion'. - title: The title of the thread. - body: The body content of the thread. - course_id: The ID of the course the thread is associated with. - anonymous: Whether the thread is posted anonymously. - anonymous_to_peers: Whether the thread is anonymous to peers. - commentable_id: The ID of the commentable entity. - at_position_list: A list of positions for @mentions. - closed: Whether the thread is closed. - context: The context of the thread, either 'course' or 'standalone'. - author_id: The ID of the author who created the thread. - author_username: The username of the author. - votes: The votes for the thread. - abuse_flaggers: A list of users who flagged the thread for abuse. - historical_abuse_flaggers: A list of users who historically flagged the thread for abuse. - closed_by: The ID of the user who closed the thread. - pinned: Whether the thread is pinned. - comments_count: The number of comments on the thread. - endorsed: Whether the thread is endorsed. - skip_timestamp_update: Whether to skip updating the timestamp (default: False). - - Returns: - int: The number of documents modified. - """ - fields = [ - ("thread_type", thread_type), - ("title", title), - ("body", body), - ("course_id", course_id), - ("anonymous", anonymous), - ("anonymous_to_peers", anonymous_to_peers), - ("commentable_id", commentable_id), - ("at_position_list", at_position_list), - ("closed", closed), - ("context", context), - ("author_id", author_id), - ("author_username", author_username), - ("votes", votes), - ("abuse_flaggers", abuse_flaggers), - ("historical_abuse_flaggers", historical_abuse_flaggers), - ("closed_by", closed_by), - ("pinned", pinned), - ("comment_count", comments_count), - ("endorsed", endorsed), - ("close_reason_code", close_reason_code), - ("closed_by_id", closed_by_id), - ("group_id", group_id), - ] - update_data: dict[str, Any] = { - field: value for field, value in fields if value is not None - } - if not closed and (close_reason_code and closed_by_id): - update_data["closed_by_id"] = None - update_data["close_reason_code"] = None - - if editing_user_id: - edit_history = [] if edit_history is None else edit_history - edit_history.append( - { - "author_id": editing_user_id, - "original_body": original_body, - "reason_code": edit_reason_code, - "editor_username": self.get_author_username(editing_user_id), - "created_at": datetime.now(), - } - ) - update_data["edit_history"] = edit_history - - if not skip_timestamp_update: - date = datetime.now() - update_data["updated_at"] = date - result = self._collection.update_one( - {"_id": ObjectId(thread_id)}, - {"$set": update_data}, - ) - - # Notify thread updated - get_handler_by_name("comment_thread_updated").send( - sender=self.__class__, comment_thread_id=thread_id - ) - return result.modified_count - - def get_author_username(self, author_id: str) -> str | None: - """Return username for the respective author_id(user_id)""" - user = Users().get(author_id) - return user.get("username") if user else None diff --git a/forum/backends/mongodb/users.py b/forum/backends/mongodb/users.py deleted file mode 100644 index 4a0b9dee..00000000 --- a/forum/backends/mongodb/users.py +++ /dev/null @@ -1,131 +0,0 @@ -"""Users Class for mongo backend.""" - -from typing import Any, Optional - -from forum.backends.mongodb.base_model import MongoBaseModel - - -class Users(MongoBaseModel): - """ - Users class for cs_comments_service user model - """ - - COLLECTION_NAME: str = "users" - - def get(self, _id: str) -> Optional[dict[str, Any]]: - """ - Get the user based on the id - """ - return self._collection.find_one({"_id": _id}) - - def insert( - self, - external_id: str, - username: Optional[str] = None, - email: Optional[str] = None, - default_sort_key: Optional[str] = "date", - read_states: Optional[list[dict[str, Any]]] = None, - course_stats: Optional[list[dict[str, Any]]] = None, - ) -> str: - """ - Inserts a new user document into the database. - - Args: - external_id: The external ID of the user. - username: The username of the user. - email: The email of the user. - default_sort_key: The default sort key for the user. - read_states: The read states of the user. - course_stats: The course statistics of the user. - - Returns: - The ID of the inserted document. - - """ - user_data: dict[str, Any] = { - "_id": external_id, - "external_id": external_id, - "username": username, - "email": email, - "default_sort_key": default_sort_key, - "read_states": read_states, - "course_stats": course_stats, - } - insert_data = {k: v for k, v in user_data.items() if v is not None} - result = self._collection.insert_one(insert_data) - return str(result.inserted_id) - - def delete(self, _id: Any) -> int: - """ - Deletes a user document from the database based on the id. - - Args: - _id: The ID of the user. - - Returns: - The number of documents deleted. - - """ - result = self._collection.delete_one({"_id": _id}) - return result.deleted_count - - def update( - self, - external_id: str, - username: Optional[str] = None, - email: Optional[str] = None, - default_sort_key: Optional[str] = None, - read_states: Optional[list[dict[str, Any]]] = None, - course_stats: Optional[list[dict[str, Any]]] = None, - ) -> int: - """ - Updates a user document in the database based on the external_id. - - Args: - external_id: The external ID of the user. - **kwargs: Keyword arguments to update the user document. - Supported keys: - - username: The new username of the user. - - email: The new email of the user. - - default_sort_key: The new default sort key for the user. - - read_states: The new read states of the user. - - course_stats: The new course statistics of the user. - - Returns: - The number of documents modified. - - """ - fields = [ - ("username", username), - ("email", email), - ("default_sort_key", default_sort_key), - ("read_states", read_states), - ("course_stats", course_stats), - ] - update_data: dict[str, Any] = { - field: value for field, value in fields if value is not None - } - - result = self._collection.update_one( - {"external_id": external_id}, - {"$set": update_data}, - ) - return result.modified_count - - def delete_read_state_by_thread_id(self, thread_id: str) -> None: - """Delete read state from users based on thread_id.""" - users = self.get_list( - **{ - "read_states.last_read_times": {"$exists": True}, - f"read_states.last_read_times.{thread_id}": {"$exists": True}, - } - ) - for user in list(users): - updated_read_states = [] - for read_state in user.get("read_states", []): - if read_state["last_read_times"].get(thread_id): - del read_state["last_read_times"][thread_id] - updated_read_states.append(read_state) - self._collection.update_one( - {"_id": user["_id"]}, {"$set": {"read_states": updated_read_states}} - ) diff --git a/forum/management/commands/forum_create_mongodb_indexes.py b/forum/management/commands/forum_create_mongodb_indexes.py deleted file mode 100644 index efa28814..00000000 --- a/forum/management/commands/forum_create_mongodb_indexes.py +++ /dev/null @@ -1,24 +0,0 @@ -"""Management command for creating mongodb indexes""" - -from django.core.management.base import BaseCommand - -from forum.backends.mongodb import BaseContents - - -class Command(BaseCommand): - """ - Django management command for creating mongodb indexes. - """ - - help = "Create or Update indexes in the mongodb for the content model" - - def handle(self, *args: list[str], **kwargs: dict[str, str]) -> None: - """ - Handles the execution of the forum_create_mongodb_indexes command. - - This command creates or updates indexes in the mongodb for the content model. - """ - BaseContents().create_indexes() - self.stdout.write( - self.style.SUCCESS("Created/Updated Mongodb indexes successfuly.") - ) diff --git a/forum/management/commands/forum_migrate_course_from_mongodb_to_mysql.py b/forum/management/commands/forum_migrate_course_from_mongodb_to_mysql.py index 298d47a4..833fd96a 100644 --- a/forum/management/commands/forum_migrate_course_from_mongodb_to_mysql.py +++ b/forum/management/commands/forum_migrate_course_from_mongodb_to_mysql.py @@ -6,7 +6,6 @@ from django.core.management.base import CommandParser from forum.migration_helpers import ( - enable_mysql_backend_for_course, get_all_course_ids, migrate_content, migrate_read_states, @@ -22,12 +21,6 @@ class Command(BaseCommand): def add_arguments(self, parser: CommandParser) -> None: """Add arguments to the command.""" - parser.add_argument( - "-T", - "--no-toggle", - action="store_true", - help="Skip course waffle flag creation", - ) parser.add_argument( "courses", nargs="+", type=str, help="List of course IDs or `all`" ) @@ -36,7 +29,6 @@ def handle(self, *args: str, **options: dict[str, Any]) -> None: """Handle the command.""" db = get_database() - create_waffle_flags = not options["no_toggle"] course_ids = list(options["courses"]) if "all" in course_ids: course_ids = get_all_course_ids(db) @@ -46,10 +38,5 @@ def handle(self, *args: str, **options: dict[str, Any]) -> None: migrate_users(db, course_id) migrate_content(db, course_id) migrate_read_states(db, course_id) - if create_waffle_flags: - enable_mysql_backend_for_course(course_id) - self.stdout.write( - f"Enabled mysql backend waffle flag for course {course_id}." - ) self.stdout.write(self.style.SUCCESS("Data migration completed successfully")) diff --git a/forum/migration_helpers.py b/forum/migration_helpers.py index 23b68e60..d51d4584 100644 --- a/forum/migration_helpers.py +++ b/forum/migration_helpers.py @@ -478,20 +478,3 @@ def log_deletion( ) -> None: """Log the deletion of a collection.""" stdout.write(f"Deleted {result.deleted_count} documents from {collection_name}") - - -# pylint: disable=import-error,import-outside-toplevel -def enable_mysql_backend_for_course(course_id: str) -> None: - """Enable MySQL backend waffle flag for a course.""" - from opaque_keys.edx.keys import CourseKey - - from openedx.core.djangoapps.waffle_utils.models import ( # type: ignore[import-not-found] - WaffleFlagCourseOverrideModel, - ) - - from forum.toggles import ENABLE_MYSQL_BACKEND - - course_key = CourseKey.from_string(course_id) - WaffleFlagCourseOverrideModel.objects.create( - course_id=course_key, waffle_flag=ENABLE_MYSQL_BACKEND.name, enabled=True - ) diff --git a/forum/search/es.py b/forum/search/es.py index 890f5351..30654059 100644 --- a/forum/search/es.py +++ b/forum/search/es.py @@ -10,10 +10,8 @@ from django.conf import settings from elasticsearch import Elasticsearch, exceptions, helpers -from forum.backends.mongodb import MODEL_INDICES as mongo_model_indices -from forum.backends.mongodb import BaseContents -from forum.backends.mongodb.threads import CommentThread from forum.backends.mysql import MODEL_INDICES as mysql_model_indices +from forum.backends.mysql.models import CommentThread from forum.constants import FORUM_MAX_DEEP_SEARCH_COMMENT_COUNT from forum.models import Content from forum.search import base @@ -48,8 +46,8 @@ class ElasticsearchModelMixin: """ @property - def models(self) -> tuple[type[BaseContents], ...]: - return mongo_model_indices + def models(self) -> tuple[type[Content], ...]: + return mysql_model_indices @property def mysql_models(self) -> tuple[type[Content], ...]: @@ -202,15 +200,6 @@ def rebuild_indices( # Create new indices and switch aliases index_names = self.create_indices() - for index_name in index_names: - current_batch = 1 - mongo_model = self.get_index_model_rel(index_name) - for response in self._import_to_es_from_mongo( - mongo_model, index_name, batch_size - ): - self.batch_import_post_process(response, current_batch) - current_batch += 1 - for index_name in index_names: current_batch = 1 mysql_model = self.get_mysql_model_from_index_name(index_name) @@ -227,28 +216,13 @@ def rebuild_indices( # Update aliases to point to new indices for index_name in index_names: - model = self.get_index_model_rel(index_name) + model = self.get_mysql_model_from_index_name(index_name) self.move_alias(model.index_name, index_name, force_delete=True) self.delete_unused_indices() log.info("Rebuild indices complete.") - def get_index_model_rel(self, index_name: str) -> BaseContents: - """ - Retrieve the model corresponding to the given index name. - - Args: - index_name (str): Name of the index. - - Returns: - model: The model associated with the index name. - """ - for model in self.models: - if index_name.startswith(model.index_name): - return model() - raise ValueError("Invalid index name") - def catchup_indices( self, index_names: list[str], start_time: datetime, batch_size: int = 100 ) -> None: @@ -260,16 +234,6 @@ def catchup_indices( start_time (datetime): The starting time for catching up. batch_size (int): Number of documents to process in each batch. """ - for index_name in index_names: - current_batch = 1 - mongo_model = self.get_index_model_rel(index_name) - mongo_query: dict[str, Any] = {"updated_at": {"$gte": start_time}} - for response in self._import_to_es_from_mongo( - mongo_model, index_name, batch_size, mongo_query - ): - self.batch_import_post_process(response, current_batch) - current_batch += 1 - for index_name in index_names: current_batch = 1 mysql_model = self.get_mysql_model_from_index_name(index_name) @@ -443,7 +407,7 @@ def initialize_indices(self, force_new_index: bool = False) -> None: if force_new_index or not self.exists_aliases(self.index_names): index_names = self.create_indices() for index_name in index_names: - model = self.get_index_model_rel(index_name) + model = self.get_mysql_model_from_index_name(index_name) self.move_alias(model.index_name, index_name, force_delete=True) else: log.info("Skipping initialization. Indices already exist.") @@ -461,7 +425,7 @@ def validate_indices(self) -> None: raise ValueError("Indices do not exist!") for index_name in actual_mappings: - model = self.get_index_model_rel(index_name) + model = self.get_mysql_model_from_index_name(index_name) expected_mapping = self.MAPPINGS[model.index_name] actual_mapping = actual_mappings[index_name]["mappings"] @@ -506,40 +470,6 @@ def exists_aliases(self, names: list[str]) -> bool: """ return all(self.exists_alias(name) for name in names) - def _import_to_es_from_mongo( - self, - model: BaseContents, - index_name: str, - batch_size: int = 500, - query: dict[str, Any] | None = None, - ) -> Iterator[tuple[int, Any]]: - """ - Import documents from the database into Elasticsearch. - - Args: - model (BaseContents): The model representing the documents. - index_name (str): The name of the index to import into. - batch_size (int): Number of documents to import in each batch. - query (dict[str, Any], optional): Query to filter documents for import. - - Yields: - Iterator[tuple[int, Any]]: Number of successful imports and any errors. - """ - cursor = model.find(query or {}).batch_size(batch_size) - actions = [] - for doc in cursor: - action = { - "_index": index_name, - "_id": str(doc.get("_id")), - "_source": model.doc_to_hash(doc), - } - actions.append(action) - if len(actions) >= batch_size: - yield helpers.bulk(self.client, actions) - actions = [] - if actions: - yield helpers.bulk(self.client, actions) - def _import_to_es_from_mysql( self, model: Any, diff --git a/forum/toggles.py b/forum/toggles.py deleted file mode 100644 index 62616cc4..00000000 --- a/forum/toggles.py +++ /dev/null @@ -1,18 +0,0 @@ -"""Forum v2 feature toggles.""" - -# pylint: disable=E0401 -from openedx.core.djangoapps.waffle_utils import CourseWaffleFlag # type: ignore[import-not-found] - -FORUM_V2_WAFFLE_FLAG_NAMESPACE = "forum_v2" - - -# .. toggle_name: forum_v2.enable_mysql_backend -# .. toggle_implementation: CourseWaffleFlag -# .. toggle_default: False -# .. toggle_description: Waffle flag to use the MySQL backend instead of Mongo backend. -# .. toggle_use_cases: temporary, open_edx -# .. toggle_creation_date: 2024-10-18 -# .. toggle_target_removal_date: 2025-06-18 -ENABLE_MYSQL_BACKEND = CourseWaffleFlag( - f"{FORUM_V2_WAFFLE_FLAG_NAMESPACE}.enable_mysql_backend", __name__ -) diff --git a/tests/conftest.py b/tests/conftest.py index b7ecc1e4..3b3c2c6b 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -11,7 +11,6 @@ from pymongo.database import Database from forum.backends.mysql.api import MySQLBackend -from forum.backends.mongodb.api import MongoBackend from test_utils.client import APIClient from test_utils.mock_es_backend import ( MockElasticsearchIndexBackend, @@ -19,16 +18,6 @@ ) -@pytest.fixture(autouse=True) -def patch_default_mongo_database(monkeypatch: pytest.MonkeyPatch) -> None: - """Mock default mongodb database for tests.""" - client: MongoClient[Any] = mongomock.MongoClient() - monkeypatch.setattr( - "forum.backends.mongodb.base_model.MongoBaseModel.MONGODB_DATABASE", - client["test_forum_db"], - ) - - @pytest.fixture(name="api_client") def fixture_api_client() -> APIClient: """Create an API client for testing.""" @@ -55,17 +44,10 @@ def mock_elasticsearch_index_backend() -> Generator[Any, Any, Any]: yield -@pytest.fixture(params=[MongoBackend, MySQLBackend]) -def patched_get_backend( - request: pytest.FixtureRequest, monkeypatch: pytest.MonkeyPatch -) -> Generator[Any, Any, Any]: - """Return the patched get_backend function for both Mongo and MySQL backends.""" - backend_class = request.param - monkeypatch.setattr( - "forum.backend.is_mysql_backend_enabled", - lambda course_id: backend_class != MongoBackend, - ) - yield backend_class +@pytest.fixture +def patched_get_backend() -> type[MySQLBackend]: + """Return MySQLBackend for tests.""" + return MySQLBackend @pytest.fixture(name="patched_mongodb") @@ -86,13 +68,3 @@ def patch_mongo_migration_database(monkeypatch: pytest.MonkeyPatch) -> Database[ lambda *args: db, ) return db - - -@pytest.fixture(autouse=True) -def patched_mongo_backend(monkeypatch: pytest.MonkeyPatch) -> Generator[Any, Any, Any]: - """Return the patched mongo_backend function for Mongo backend.""" - monkeypatch.setattr( - "forum.backend.is_mysql_backend_enabled", - lambda course_id: False, - ) - yield MongoBackend diff --git a/tests/e2e/conftest.py b/tests/e2e/conftest.py index 7ee86d5c..7dc04606 100644 --- a/tests/e2e/conftest.py +++ b/tests/e2e/conftest.py @@ -3,20 +3,14 @@ """ import logging -import time -import typing as t import pytest -from pymongo.errors import ServerSelectionTimeoutError -from forum.mongo import get_database +from forum.backends.mysql.api import MySQLBackend as patched_mysql_backend from test_utils.client import APIClient log = logging.getLogger(__name__) -MONGO_TIMEOUT = 60 -MONGO_SLEEP_INTERVAL = 5 - @pytest.fixture(name="api_client") def fixture_api_client() -> APIClient: @@ -24,38 +18,6 @@ def fixture_api_client() -> APIClient: return APIClient() -def wait_for_mongodb() -> None: - """Wait for MongoDB to start.""" - db = get_database() - timeout = MONGO_TIMEOUT - while timeout > 0: - try: - db.command("ping") - log.info("Connected to the MongoDB") - return - except ServerSelectionTimeoutError: - log.info("Waiting for mongodb to connect") - time.sleep(MONGO_SLEEP_INTERVAL) - timeout -= MONGO_SLEEP_INTERVAL - raise Exception("Elasticsearch did not start in time") - - -@pytest.fixture(autouse=True) -def mongo_cleanup() -> None: - """Cleanup MongoDB collections after each test.""" - wait_for_mongodb() - db = get_database() - - # Clean up collections after each test case - for collection_name in db.list_collection_names(): - db.drop_collection(collection_name) - - -@pytest.fixture(autouse=True) -def patch_default_mongo_database() -> None: - """Override the patch statement.""" - - @pytest.fixture(autouse=True) def mock_elasticsearch_document_backend() -> None: """Mock again the mocked backend to restore the actual backend.""" @@ -67,9 +29,9 @@ def mock_elasticsearch_index_backend() -> None: @pytest.fixture(name="user_data") -def create_test_user(patched_get_backend: t.Any) -> tuple[str, str]: +def create_test_user() -> tuple[str, str]: """Create a user.""" - backend = patched_get_backend() + backend = patched_mysql_backend() user_id = "1" username = "test_user" diff --git a/tests/e2e/docker-compose.yml b/tests/e2e/docker-compose.yml index 791d347e..14cca4b8 100644 --- a/tests/e2e/docker-compose.yml +++ b/tests/e2e/docker-compose.yml @@ -1,12 +1,6 @@ # Docker Compose service declaration for end-to-end testing. This runs services without # data persistence. Be careful about losing your data! services: - mongodb: - # https://hub.docker.com/_/mongo/tags - image: docker.io/mongo:7.0.14 - ports: - - 127.0.0.1:27017:27017 - elasticsearch: # https://hub.docker.com/_/elasticsearch/tags image: docker.io/elasticsearch:7.17.23 diff --git a/tests/test_backends/test_mongodb/test_comments.py b/tests/test_backends/test_mongodb/test_comments.py deleted file mode 100644 index d35d64c4..00000000 --- a/tests/test_backends/test_mongodb/test_comments.py +++ /dev/null @@ -1,94 +0,0 @@ -#!/usr/bin/env python -""" -Tests for the `Comment` model. -""" - -from forum.backends.mongodb import Comment - - -def test_insert() -> None: - """Test insert a comment into MongoDB.""" - comment_id = Comment().insert( - body="

This is a test comment

", - course_id="course1", - comment_thread_id="66af33634a1e1f001b7ed57f", - author_id="author1", - author_username="author_user", - ) - assert comment_id is not None - comment_data = Comment().get(_id=comment_id) - assert comment_data is not None - assert comment_data["body"] == "

This is a test comment

" - - -def test_delete() -> None: - """Test delete a comment from MongoDB.""" - comment_id = Comment().insert( - body="

This is a test comment

", - course_id="course1", - comment_thread_id="66af33634a1e1f001b7ed57f", - author_id="author1", - author_username="author_user", - ) - - invalid_id = "66dedf65a2e0d02feebde812" - result = Comment().delete(invalid_id) - assert result == 0 - - result = Comment().delete(comment_id) - assert result == 1 - comment_data = Comment().get(_id=comment_id) - assert comment_data is None - - -def test_list() -> None: - """Test list all comments from MongoDB.""" - course_id = "course-xyz" - thread_id = "66af33634a1e1f001b7ed57f" - author_id = "4" - author_username = "edly" - - Comment().insert( - "

Comment 1

", - course_id, - author_id, - comment_thread_id=thread_id, - author_username=author_username, - ) - Comment().insert( - "

Comment 2

", - course_id, - author_id, - comment_thread_id=thread_id, - author_username=author_username, - ) - Comment().insert( - "

Comment 3

", - course_id, - author_id, - comment_thread_id=thread_id, - author_username=author_username, - ) - - comments_list = Comment().get_list() - assert len(list(comments_list)) == 3 - assert all(comment["body"].startswith("

Comment") for comment in comments_list) - - -def test_update() -> None: - """Test update a comment in MongoDB.""" - comment_id = Comment().insert( - body="

This is a test comment

", - course_id="course1", - comment_thread_id="66af33634a1e1f001b7ed57f", - author_id="author1", - author_username="author_user", - ) - - result = Comment().update( - comment_id=comment_id, - body="

Updated comment

", - ) - assert result == 1 - comment_data = Comment().get(_id=comment_id) or {} - assert comment_data.get("body", "") == "

Updated comment

" diff --git a/tests/test_backends/test_mongodb/test_mongo_api.py b/tests/test_backends/test_mongodb/test_mongo_api.py deleted file mode 100644 index c1ed2e23..00000000 --- a/tests/test_backends/test_mongodb/test_mongo_api.py +++ /dev/null @@ -1,55 +0,0 @@ -""" -Test suit for MongoDB backend API functionalities. -""" - -import unittest - -from forum.backends.mongodb import CommentThread -from forum.backends.mongodb.api import MongoBackend -from forum.serializers.thread import ThreadSerializer - - -class TestMongoAPI(unittest.TestCase): - """ - Test suite for MongoDB backend API functionalities. - """ - - def setUp(self) -> None: - self.thread_1 = CommentThread().insert( - "Thread 1", - "Body 1", - "course_id", - "id_1", - "1", - "user1", - ) - self.thread_2 = CommentThread().insert( - "Thread 2", - "Body 2", - "course_id", - "id_2", - "1", - "user1", - ) - self.thread_3 = CommentThread().insert( - "Thread 3", - "Body 3", - "course_id", - "id_2", - "user1", - ) - - def test_filter_by_commentable_ids(self) -> None: - """ - Test filtering threads by commentable_ids. - """ - threads = MongoBackend.get_threads( - user_id="", - params={"commentable_ids": ["id_2"], "course_id": "course_id"}, - serializer=ThreadSerializer, - thread_ids=[self.thread_1, self.thread_2, self.thread_3], - ) - # make sure the threads are filtered correctly by commentable_ids aka Topics ids - assert threads["thread_count"] == 2 - for thread in threads["collection"]: - assert thread["commentable_id"] == "id_2" diff --git a/tests/test_backends/test_mongodb/test_subscriptions.py b/tests/test_backends/test_mongodb/test_subscriptions.py deleted file mode 100644 index 7b07dd18..00000000 --- a/tests/test_backends/test_mongodb/test_subscriptions.py +++ /dev/null @@ -1,101 +0,0 @@ -#!/usr/bin/env python -""" -Tests for the Subscriptions model. -""" - -from forum.backends.mongodb import Subscriptions - - -def test_get() -> None: - """Test get subscription from mongodb""" - subscriber_id = "test_subscriber_id" - source_id = "test_source_id" - source_type = "test_source_type" - Subscriptions().insert( - subscriber_id, - source_id, - source_type, - ) - subscription_data = Subscriptions().get_subscription(subscriber_id, source_id) - assert subscription_data is not None - assert subscription_data["subscriber_id"] == subscriber_id - assert subscription_data["source_id"] == source_id - assert subscription_data["source_type"] == source_type - - -def test_insert() -> None: - """Test insert subscription from mongodb""" - subscriber_id = "test_subscriber_id" - source_id = "test_source_id" - source_type = "test_source_type" - result = Subscriptions().insert(subscriber_id, source_id, source_type) - assert result is not None - subscription_data = Subscriptions().get_subscription(subscriber_id, source_id) - assert subscription_data is not None - assert subscription_data["subscriber_id"] == subscriber_id - assert subscription_data["source_id"] == source_id - assert subscription_data["source_type"] == source_type - - -def test_delete() -> None: - """Test delete subscription from mongodb""" - subscriber_id = "test_subscriber_id" - source_id = "test_source_id" - source_type = "test_source_type" - Subscriptions().insert(subscriber_id, source_id, source_type) - result = Subscriptions().delete_subscription(subscriber_id, source_id) - assert result == 1 - subscription_data = Subscriptions().get_subscription(subscriber_id, source_id) - assert subscription_data is None - - -def test_list() -> None: - """Test list subscription from mongodb""" - Subscriptions().insert( - subscriber_id="user1", - source_id="source1", - source_type="type1", - ) - Subscriptions().insert( - subscriber_id="user2", - source_id="source2", - source_type="type2", - ) - Subscriptions().insert( - subscriber_id="user3", - source_id="source3", - source_type="type3", - ) - subscriptions_list = Subscriptions().get_list() - assert len(list(subscriptions_list)) == 3 - assert all( - subscription["subscriber_id"] in ["user1", "user2", "user3"] - for subscription in subscriptions_list - ) - - -def test_update() -> None: - """Test update subscription from mongodb""" - subscriber_id = "test_subscriber_id" - source_id = "test_source_id" - source_type = "test_source_type" - Subscriptions().insert( - subscriber_id=subscriber_id, - source_id=source_id, - source_type=source_type, - ) - - new_source_type = "new_source_type" - result = Subscriptions().update( - subscriber_id, - source_id, - source_type=new_source_type, - ) - assert result is not None - assert result == 1 - - subscription_data = Subscriptions().get_subscription(subscriber_id, source_id) - assert subscription_data is not None - assert subscription_data["subscriber_id"] == subscriber_id - assert subscription_data["source_id"] == source_id - assert subscription_data["source_type"] == new_source_type diff --git a/tests/test_backends/test_mongodb/test_threads.py b/tests/test_backends/test_mongodb/test_threads.py deleted file mode 100644 index 01ca228b..00000000 --- a/tests/test_backends/test_mongodb/test_threads.py +++ /dev/null @@ -1,120 +0,0 @@ -#!/usr/bin/env python -""" -Tests for the `CommentThread` model. -""" - -import pytest - -from forum.backends.mongodb import CommentThread - - -def test_insert_invalid_data() -> None: - """The inserting invalid data""" - with pytest.raises(ValueError, match="Invalid thread_type"): - CommentThread().insert( - title="Test title", - body="Test body", - course_id="course_123", - commentable_id="commentable_123", - author_id="author_123", - thread_type="invalid_type", - ) - - with pytest.raises(ValueError, match="Invalid context"): - CommentThread().insert( - title="Test title", - body="Test body", - course_id="course_123", - commentable_id="commentable_123", - author_id="author_123", - context="invalid_context", - ) - - -def test_insert() -> None: - """Test insert a comment thread into MongoDB.""" - thread_id = CommentThread().insert( - title="Test Thread", - body="This is a test thread", - course_id="course1", - commentable_id="commentable1", - author_id="author1", - author_username="author_user", - ) - assert thread_id is not None - thread_data = CommentThread().get(thread_id) - assert thread_data is not None - assert thread_data["title"] == "Test Thread" - assert thread_data["body"] == "This is a test thread" - - -def test_delete() -> None: - """Test delete a comment thread from MongoDB.""" - thread_id = CommentThread().insert( - title="Test Thread", - body="This is a test thread", - course_id="course1", - commentable_id="commentable1", - author_id="author1", - author_username="author_user", - ) - result = CommentThread().delete(thread_id) - assert result == 1 - thread_data = CommentThread().get(thread_id) - assert thread_data is None - - -def test_list() -> None: - """Test list all comment threads from MongoDB.""" - CommentThread().insert( - "Thread 1", - "Body 1", - "_type", - "CommentThread", - "1", - "user1", - ) - CommentThread().insert( - "Thread 2", - "Body 2", - "_type", - "CommentThread", - "1", - "user1", - ) - CommentThread().insert( - "Thread 3", - "Body 3", - "_type", - "CommentThread", - "1", - "user1", - ) - threads_list = CommentThread().get_list() - assert len(list(threads_list)) == 3 - assert all(thread["title"].startswith("Thread") for thread in threads_list) - - -def test_update() -> None: - """Test update a comment thread in MongoDB.""" - thread_id = CommentThread().insert( - title="Test Thread", - body="This is a test thread", - course_id="course1", - commentable_id="commentable1", - author_id="author1", - author_username="author_user", - ) - - result = CommentThread().update( - thread_id=thread_id, - title="Updated Title", - body="Updated body", - commentable_id="new_commentable_id", - ) - assert result == 1 - thread_data = CommentThread().get(thread_id) - assert thread_data is not None - assert thread_data["title"] == "Updated Title" - assert thread_data["body"] == "Updated body" - assert thread_data["commentable_id"] == "new_commentable_id" diff --git a/tests/test_backends/test_mongodb/test_users.py b/tests/test_backends/test_mongodb/test_users.py deleted file mode 100644 index d53a40ef..00000000 --- a/tests/test_backends/test_mongodb/test_users.py +++ /dev/null @@ -1,95 +0,0 @@ -#!/usr/bin/env python -""" -Tests for the `forum` models module. -""" - -from forum.backends.mongodb import Users - - -def test_get() -> None: - """Test get user from mongodb""" - external_id = "test_external_id" - username = "test_username" - email = "test_email" - Users().insert( - external_id, - username, - email, - ) - user_data = Users().get(external_id) - assert user_data is not None - assert user_data["_id"] == external_id - assert user_data["external_id"] == external_id - assert user_data["username"] == username - assert user_data["email"] == email - - -def test_insert() -> None: - """Test insert user from mongodb""" - external_id = "test_external_id" - username = "test_username" - email = "test_email" - result = Users().insert(external_id, username, email) - assert result is not None - user_data = Users().get(external_id) - assert user_data is not None - assert user_data["_id"] == external_id - assert user_data["external_id"] == external_id - assert user_data["username"] == username - assert user_data["email"] == email - - -def test_delete() -> None: - """Test delete user from mongodb""" - external_id = "test_external_id" - Users().insert(external_id, "test_username", "test_email") - result = Users().delete(external_id) - assert result == 1 - user_data = Users().get(external_id) - assert user_data is None - - -def test_list() -> None: - """Test list user from mongodb""" - Users().insert( - external_id="user1", - username="user1", - email="user1", - ) - Users().insert( - external_id="user2", - username="user2", - email="user1", - ) - Users().insert( - external_id="user3", - username="user3", - email="user1", - ) - users_list = Users().get_list() - assert len(list(users_list)) == 3 - assert all(user["username"] in ["user1", "user2", "user3"] for user in users_list) - - -def test_update() -> None: - """Test update user from mongodb""" - external_id = "test_external_id" - username = "test_username" - email = "test_email" - Users().insert( - external_id=external_id, - username=username, - email=email, - ) - - new_username = "new_username" - new_email = "new_email" - result = Users().update(external_id, username=new_username, email=new_email) - assert result is not None - assert result == 1 - - user_data = Users().get(external_id) - assert user_data is not None - assert user_data["external_id"] == external_id - assert user_data["username"] == new_username - assert user_data["email"] == new_email diff --git a/tests/test_management/test_commands/test_migration_commands.py b/tests/test_management/test_commands/test_migration_commands.py index 49e48f48..bef8400a 100644 --- a/tests/test_management/test_commands/test_migration_commands.py +++ b/tests/test_management/test_commands/test_migration_commands.py @@ -27,15 +27,6 @@ pytestmark = pytest.mark.django_db -@pytest.fixture(autouse=True) -def patch_enable_mysql_backend(monkeypatch: pytest.MonkeyPatch) -> None: - """Patch enable_mysql_backend_for_course to just return.""" - monkeypatch.setattr( - "forum.migration_helpers.enable_mysql_backend_for_course", - lambda course_id: None, - ) - - def test_migrate_users(patched_mongodb: Database[Any]) -> None: patched_mongodb.users.insert_one( { diff --git a/tests/test_views/test_threads.py b/tests/test_views/test_threads.py index 46892158..dd4fe47f 100644 --- a/tests/test_views/test_threads.py +++ b/tests/test_views/test_threads.py @@ -1,12 +1,9 @@ """Test threads api endpoints.""" import time -from datetime import datetime from typing import Any, Optional import pytest -from forum.backends.mongodb.api import MongoBackend -from forum.backends.mongodb.users import Users from test_utils.client import APIClient pytestmark = pytest.mark.django_db @@ -891,56 +888,6 @@ def test_response_for_thread_type_question( assert thread["non_endorsed_resp_total"] == 1 -def test_read_states_deletion_of_a_thread_on_thread_deletion( - api_client: APIClient, patched_mongo_backend: MongoBackend -) -> None: - """Test delete read_states of the thread on deletion of a thread for mongodb.""" - user_id, thread_id = setup_models(backend=patched_mongo_backend) - comment_id_1, comment_id_2 = create_comments_in_a_thread( - patched_mongo_backend, thread_id - ) - thread_from_db = patched_mongo_backend.get_thread(thread_id) - assert thread_from_db is not None - assert thread_from_db["comment_count"] == 2 - get_thread_response = api_client.get_json( - f"/api/v2/threads/{thread_id}", - params={ - "recursive": False, - "with_responses": True, - "user_id": int(user_id), - "mark_as_read": False, - "resp_skip": 0, - "resp_limit": 10, - "reverse_order": "true", - "merge_question_type_responses": False, - }, - ) # call get_thread API to save read_states of this thread in user model - assert get_thread_response.status_code == 200 - assert is_thread_id_exists_in_user_read_state(user_id, thread_id) is True - response = api_client.delete_json(f"/api/v2/threads/{thread_id}") - assert response.status_code == 200 - assert patched_mongo_backend.get_thread(thread_id) is None - assert patched_mongo_backend.get_comment(comment_id_1) is None - assert patched_mongo_backend.get_comment(comment_id_2) is None - assert ( - patched_mongo_backend.get_subscription( - subscriber_id=user_id, source_id=thread_id - ) - is None - ) - assert is_thread_id_exists_in_user_read_state(user_id, thread_id) is False - - -def is_thread_id_exists_in_user_read_state(user_id: str, thread_id: str) -> bool: - """Return True or False if thread_id exists in read_states of any user.""" - user = Users().find_one({"_id": user_id}) - if user: - for read_state in user.get("read_states", []): - if thread_id in read_state.get("last_read_times", {}): - return True - return False - - def test_filter_by_group_id(api_client: APIClient, patched_get_backend: Any) -> None: """ Filter threads by their group_id. This should return: @@ -1072,106 +1019,3 @@ def test_pagination_in_thread_comments( assert len(thread["non_endorsed_responses"]) == comments_count - resp_limit assert thread["resp_skip"] == resp_limit assert thread["resp_limit"] == resp_limit - - -def test_read_states_deletion_on_thread_deletion_without_read_states( - api_client: APIClient, patched_mongo_backend: MongoBackend -) -> None: - """Test delete read_states of the thread on deletion of a thread when there are no read states.""" - user_id, thread_id = setup_models(backend=patched_mongo_backend) - comment_id_1, comment_id_2 = create_comments_in_a_thread( - patched_mongo_backend, thread_id - ) - thread_from_db = patched_mongo_backend.get_thread(thread_id) - assert thread_from_db is not None - assert thread_from_db["comment_count"] == 2 - assert is_thread_id_exists_in_user_read_state(user_id, thread_id) is False - - response = api_client.delete_json(f"/api/v2/threads/{thread_id}") - assert response.status_code == 200 - assert patched_mongo_backend.get_thread(thread_id) is None - assert patched_mongo_backend.get_comment(comment_id_1) is None - assert patched_mongo_backend.get_comment(comment_id_2) is None - assert ( - patched_mongo_backend.get_subscription( - subscriber_id=user_id, source_id=thread_id - ) - is None - ) - assert is_thread_id_exists_in_user_read_state(user_id, thread_id) is False - - -def test_read_states_deletion_on_thread_deletion_with_multiple_read_states( - api_client: APIClient, patched_mongo_backend: MongoBackend -) -> None: - """Test delete read_states of the thread on deletion of a thread when there are multiple read states.""" - # Setup first thread and read state - user_id_1, thread_id_1 = setup_models(backend=patched_mongo_backend) - get_thread_response = api_client.get_json( - f"/api/v2/threads/{thread_id_1}", - params={ - "recursive": False, - "with_responses": True, - "user_id": int(user_id_1), - "mark_as_read": True, - "resp_skip": 0, - "resp_limit": 10, - "reverse_order": "true", - "merge_question_type_responses": False, - }, - ) - assert get_thread_response.status_code == 200 - assert is_thread_id_exists_in_user_read_state(user_id_1, thread_id_1) is True - - # Setup second thread and read state - user_id_2, thread_id_2 = setup_models( - backend=patched_mongo_backend, - user_id="2", - username="user2", - course_id="course2", - ) - get_thread_response = api_client.get_json( - f"/api/v2/threads/{thread_id_2}", - params={ - "recursive": False, - "with_responses": True, - "user_id": int(user_id_2), - "mark_as_read": True, - "resp_skip": 0, - "resp_limit": 10, - "reverse_order": "true", - "merge_question_type_responses": False, - }, - ) - assert get_thread_response.status_code == 200 - assert is_thread_id_exists_in_user_read_state(user_id_2, thread_id_2) is True - - # Delete first thread and verify its read state is removed while second remains - response = api_client.delete_json(f"/api/v2/threads/{thread_id_1}") - assert response.status_code == 200 - assert patched_mongo_backend.get_thread(thread_id_1) is None - assert is_thread_id_exists_in_user_read_state(user_id_1, thread_id_1) is False - assert is_thread_id_exists_in_user_read_state(user_id_2, thread_id_2) is True - - -def test_read_states_deletion_checks_thread_id_existence( - api_client: APIClient, patched_mongo_backend: MongoBackend -) -> None: - """Test that read state deletion only occurs when thread_id exists in last_read_times.""" - user_id, thread_id = setup_models(backend=patched_mongo_backend) - - other_thread_id = "other_thread_id" - read_states = [ - { - "course_id": "course1", - "last_read_times": {other_thread_id: datetime.now()}, - } - ] - patched_mongo_backend.update_user(user_id, {"read_states": read_states}) - - assert is_thread_id_exists_in_user_read_state(user_id, other_thread_id) is True - assert is_thread_id_exists_in_user_read_state(user_id, thread_id) is False - - response = api_client.delete_json(f"/api/v2/threads/{thread_id}") - assert response.status_code == 200 - assert is_thread_id_exists_in_user_read_state(user_id, other_thread_id) is True