Merge pull request #51 from microsoft/dev

Merge dev to master for 1.0rc1 release

Author: absci

.gitignore

@@ -6,7 +6,7 @@
 Thumbs.db
 
 *.egg-info
-
+*.dll
 tests/local_settings.py
 
 # Virtual Env

CONTRIBUTING.md

@@ -2,6 +2,39 @@
 
 ## How to contribute
 
+### Run unit tests
+After changes made to the project, it's a good idea to run the unit tests before making a pull request. 
+
+
+1. **Run SQL Server**  
+   Make sure you have SQL Server running in your local machine. 
+   Download and install SQL Server [here](https://www.microsoft.com/en-us/sql-server/sql-server-downloads), or you could use docker. Change `testapp/settings.py` to match your SQL Server login username and password.  
+   
+   ```
+   docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=MyPassword42' -p 1433:1433 -d mcr.microsoft.com/mssql/server:2019-latest
+   ```
+2. **Clone Django**   
+   In `mssql-django` folder. 
+   ```
+   # Install your local mssql-django
+   pip install -e .
+
+   # The unit test suite are in `Django` folder, so we need to clone it
+   git clone https://github.com/django/django.git --depth 1
+   ```
+   
+3. **Install Tox**  
+   ```
+   # we use `tox` to run tests and install dependencies
+   pip install tox
+   ``` 
+4. **Run Tox**  
+   ```
+   # eg. run django 3.1 tests with Python 3.7
+   tox -e py37-django31
+   ```
+
+---
 This project welcomes contributions and suggestions.  Most contributions require you to agree to a
 Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
 the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

LICENSE.txt

@@ -1,11 +1,34 @@
 Project Name: mssql-django
 
-MIT License
+BSD 3-Clause License
 
-Copyright (c) Microsoft Corporation.
+Copyright (c) 2021, Microsoft Corporation
+              2019, ES Solutions AB
+              2018, Michiya Takahashi
+              2008, 2009 django-pyodbc developers
+All rights reserved.
 
-Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
 
-The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+* Redistributions of source code must retain the above copyright notice, this
+  list of conditions and the following disclaimer.
 
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE
+* Redistributions in binary form must reproduce the above copyright notice,
+  this list of conditions and the following disclaimer in the documentation
+  and/or other materials provided with the distribution.
+
+* Neither the name of the copyright holder nor the names of its
+  contributors may be used to endorse or promote products derived from
+  this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

README.md

@@ -4,13 +4,13 @@ Welcome to the MSSQL-Django 3rd party backend project!
 
 *mssql-django* is a fork of [django-mssql-backend](https://pypi.org/project/django-mssql-backend/). This project provides an enterprise database connectivity option for the Django Web Framework, with support for Microsoft SQL Server and Azure SQL Database.
 
-We'd like to give thanks to the community that made this project possible, with particular recognition of the contributors: OskarPersson, michiya, dlo and the original Google Code django-pyodbc team. Moving forward we encourage partipation in this project from both old and new contributors! 
+We'd like to give thanks to the community that made this project possible, with particular recognition of the contributors: OskarPersson, michiya, dlo and the original Google Code django-pyodbc team. Moving forward we encourage partipation in this project from both old and new contributors!
 
 We hope you enjoy using the MSSQL-Django 3rd party backend.
 
 ## Features
 
--  Supports Django 2.2, 3.0
+-  Supports Django 2.2, 3.0 and 3.1
 -  Tested on Microsoft SQL Server 2016, 2017, 2019
 -  Passes most of the tests of the Django test suite
 -  Compatible with
@@ -20,12 +20,12 @@ We hope you enjoy using the MSSQL-Django 3rd party backend.
 
 ## Dependencies
 
--  Django 2.2 or 3.0
+-  Django 2.2, 3.0 or 3.1
 -  pyodbc 3.0 or newer
 
 ## Installation
 
-1. Install pyodbc 3.0 (or newer) and Django 2.2 (or 3.0)
+1. Install pyodbc 3.0 (or newer) and Django 2.2, 3.0 or 3.1
 
 2. Install mssql-django:
 
@@ -74,14 +74,18 @@ in DATABASES control the behavior of the backend:
    Boolean. Set this to `False` if you want to disable
    Django's transaction management and implement your own.
 
+-  Trusted_Connection
+
+   String. Default is `"yes"`. Can be set to `"no"` if required.
+
 and the following entries are also available in the `TEST` dictionary
 for any given database-level settings dictionary:
 
 -  NAME
 
    String. The name of database to use when running the test suite.
    If the default value (`None`) is used, the test database will use
-   the name `"test\_" + NAME`.
+   the name `"test_" + NAME`.
 
 -  COLLATION
 
@@ -135,10 +139,12 @@ Dictionary. Current available keys are:
    definition present in the ``freetds.conf`` FreeTDS configuration file
    instead of a hostname or an IP address.
 
-   But if this option is present and it's value is ``True``, this
-   special behavior is turned off.
+   But if this option is present and its value is ``True``, this
+   special behavior is turned off. Instead, connections to the database
+   server will be established using ``HOST`` and ``PORT`` options, without
+   requiring ``freetds.conf`` to be configured.
 
-   See http://www.freetds.org/userguide/dsnless.htm for more information.
+   See https://www.freetds.org/userguide/dsnless.html for more information.
 
 -  unicode_results
 
@@ -149,7 +155,7 @@ Dictionary. Current available keys are:
 -  extra_params
 
    String. Additional parameters for the ODBC connection. The format is
-   ``"param=value;param=value"``.
+   ``"param=value;param=value"``, [Azure AD Authentication](https://github.com/microsoft/mssql-django/wiki/Azure-AD-Authentication) can be added to this field.
 
 -  collation
 
@@ -216,7 +222,9 @@ Here is an example of the database settings:
 
 The following features are currently not supported:
 - mssql-django does not support SQL-based regex commands
-- Altering a model field from or to AutoField at migration
+- Altering a model field from or to AutoField at migration  
+
+Certain limitations for JSONField lookups, more details [here](https://github.com/microsoft/mssql-django/wiki/JSONField).
 
 ## Future Plans
 
@@ -225,6 +233,8 @@ The following features and additions are planned:
 
 ## Contributing
 
+More details on contributing can be found [Here](CONTRIBUTING.md).
+
 This project welcomes contributions and suggestions.  Most contributions require you to agree to a
 Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
 the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

azure-pipelines.yml

@@ -14,68 +14,28 @@ schedules:
   always: true
 
 jobs:
-  - job: Windows
+  - job: Linux
     pool:
-      Django-agent-pool
+      vmImage: ubuntu-18.04
 
     strategy:
       matrix:
-        Python 3.8 - Django 3.1:
+        Python 3.9 - Django 3.2:
+          python.version: '3.9'
+          tox.env: 'py39-django32'
+        Python 3.8 - Django 3.2:
           python.version: '3.8'
-          tox.env: 'py38-django31'
-        Python 3.7 - Django 3.1:
+          tox.env: 'py38-django32'
+        Python 3.7 - Django 3.2:
           python.version: '3.7'
-          tox.env: 'py37-django31'
-        Python 3.6 - Django 3.1:
+          tox.env: 'py37-django32'
+        Python 3.6 - Django 3.2:
           python.version: '3.6'
-          tox.env: 'py36-django31'
+          tox.env: 'py36-django32'
 
-        Python 3.8 - Django 3.0:
-          python.version: '3.8'
-          tox.env: 'py38-django30'
-        Python 3.7 - Django 3.0:
-          python.version: '3.7'
-          tox.env: 'py37-django30'
-        Python 3.6 - Django 3.0:
-          python.version: '3.6'
-          tox.env: 'py36-django30'
-
-        Python 3.7 - Django 2.2:
-          python.version: '3.7'
-          tox.env: 'py37-django22'
-        Python 3.6 - Django 2.2:
-          python.version: '3.6'
-          tox.env: 'py36-django22'
-
-    steps:
-      - task: CredScan@2
-        inputs:
-          toolMajorVersion: 'V2'
-      - task: UsePythonVersion@0
-        inputs:
-          versionSpec: "$(python.version)"
-        displayName: Use Python $(python.version)
-
-      - script: |
-          python -m pip install --upgrade pip wheel setuptools
-          pip install tox
-          git clone https://github.com/django/django.git
-        displayName: Install requirements
-
-      - script: tox -e $(tox.env)
-        displayName: Run tox
-
-      - task: PublishCodeCoverageResults@1
-        inputs:
-          codeCoverageTool: 'Cobertura'
-          summaryFileLocation: 'C:\Windows\ServiceProfiles\NetworkService\coverage.xml'
-
-  - job: Linux
-    pool:
-      vmImage: ubuntu-18.04
-
-    strategy:
-      matrix:
+        Python 3.9 - Django 3.1:
+          python.version: '3.9'
+          tox.env: 'py39-django31'
         Python 3.8 - Django 3.1:
           python.version: '3.8'
           tox.env: 'py38-django31'
@@ -86,6 +46,9 @@ jobs:
           python.version: '3.6'
           tox.env: 'py36-django31'
 
+        Python 3.9 - Django 3.0:
+          python.version: '3.9'
+          tox.env: 'py39-django30'
         Python 3.8 - Django 3.0:
           python.version: '3.8'
           tox.env: 'py38-django30'
@@ -131,3 +94,10 @@ jobs:
         inputs:
           codeCoverageTool: 'Cobertura'
           summaryFileLocation: '/home/vsts/coverage.xml'
+
+      - task: PublishTestResults@2
+        displayName: Publish test results via jUnit
+        inputs:
+          testResultsFormat: 'JUnit'
+          testResultsFiles: '/home/vsts/result.xml'
+          testRunTitle: 'junit-$(Agent.OS)-$(Agent.OSArchitecture)-$(tox.env)'

manage.py

@@ -1,7 +1,8 @@
+#!/usr/bin/env python
+
 # Copyright (c) Microsoft Corporation.
-# Licensed under the MIT license.
+# Licensed under the BSD license.
 
-#!/usr/bin/env python
 import os
 import sys
 

mssql/__init__.py

@@ -1,4 +1,4 @@
 # Copyright (c) Microsoft Corporation.
-# Licensed under the MIT license.
+# Licensed under the BSD license.
 
 import mssql.functions  # noqa

mssql/base.py

@@ -1,5 +1,5 @@
 # Copyright (c) Microsoft Corporation.
-# Licensed under the MIT license.
+# Licensed under the BSD license.
 
 """
 MS SQL Server database backend for Django.
@@ -75,7 +75,7 @@ class DatabaseWrapper(BaseDatabaseWrapper):
         'AutoField': 'int',
         'BigAutoField': 'bigint',
         'BigIntegerField': 'bigint',
-        'BinaryField': 'varbinary(max)',
+        'BinaryField': 'varbinary(%(max_length)s)',
         'BooleanField': 'bit',
         'CharField': 'nvarchar(%(max_length)s)',
         'DateField': 'date',
@@ -88,10 +88,12 @@ class DatabaseWrapper(BaseDatabaseWrapper):
         'IntegerField': 'int',
         'IPAddressField': 'nvarchar(15)',
         'GenericIPAddressField': 'nvarchar(39)',
+        'JSONField': 'nvarchar(max)',
         'NullBooleanField': 'bit',
         'OneToOneField': 'int',
         'PositiveIntegerField': 'int',
         'PositiveSmallIntegerField': 'smallint',
+        'PositiveBigIntegerField' : 'bigint',
         'SlugField': 'nvarchar(%(max_length)s)',
         'SmallAutoField': 'smallint',
         'SmallIntegerField': 'smallint',
@@ -105,8 +107,10 @@ class DatabaseWrapper(BaseDatabaseWrapper):
         'SmallAutoField': 'IDENTITY (1, 1)',
     }
     data_type_check_constraints = {
+        'JSONField': '(ISJSON ("%(column)s") = 1)',
         'PositiveIntegerField': '[%(column)s] >= 0',
         'PositiveSmallIntegerField': '[%(column)s] >= 0',
+        'PositiveBigIntegerField': '[%(column)s] >= 0',
     }
     operators = {
         # Since '=' is used not only for string comparision there is no way
@@ -249,9 +253,10 @@ def get_new_connection(self, conn_params):
         user = conn_params.get('USER', None)
         password = conn_params.get('PASSWORD', None)
         port = conn_params.get('PORT', None)
+        trusted_connection = conn_params.get('Trusted_Connection', 'yes')
 
         options = conn_params.get('OPTIONS', {})
-        driver = options.get('driver', 'ODBC Driver 13 for SQL Server')
+        driver = options.get('driver', 'ODBC Driver 17 for SQL Server')
         dsn = options.get('dsn', None)
 
         # Microsoft driver names assumed here are:
@@ -286,10 +291,11 @@ def get_new_connection(self, conn_params):
 
         if user:
             cstr_parts['UID'] = user
-            cstr_parts['PWD'] = password
+            if 'Authentication=ActiveDirectoryInteractive' not in options.get('extra_params', ''):
+                cstr_parts['PWD'] = password
         else:
             if ms_drivers.match(driver):
-                cstr_parts['Trusted_Connection'] = 'yes'
+                cstr_parts['Trusted_Connection'] = trusted_connection
             else:
                 cstr_parts['Integrated Security'] = 'SSPI'
 
@@ -487,12 +493,12 @@ def check_constraints(self, table_names=None):
 
     def disable_constraint_checking(self):
         if not self.needs_rollback:
-            self.cursor().execute('EXEC sp_msforeachtable "ALTER TABLE ? NOCHECK CONSTRAINT ALL"')
+            self._execute_foreach('ALTER TABLE %s NOCHECK CONSTRAINT ALL')
         return not self.needs_rollback
 
     def enable_constraint_checking(self):
         if not self.needs_rollback:
-            self.cursor().execute('EXEC sp_msforeachtable "ALTER TABLE ? WITH CHECK CHECK CONSTRAINT ALL"')
+            self._execute_foreach('ALTER TABLE %s WITH NOCHECK CHECK CONSTRAINT ALL')
 
 
 class CursorWrapper(object):

mssql/client.py

@@ -1,5 +1,5 @@
 # Copyright (c) Microsoft Corporation.
-# Licensed under the MIT license.
+# Licensed under the BSD license.
 
 import re
 import subprocess