diff --git a/Changelog.md b/Changelog.md index 3b44916cdc..7944e610a9 100644 --- a/Changelog.md +++ b/Changelog.md @@ -1,4 +1,7 @@ # FOSSA CLI Changelog +## 3.15.1 +- Remove `fossa snippets` subcommand and documentation ([#1623](https://github.com/fossas/fossa-cli/pull/1623)) + ## 3.15.0 - Make snippet scanning non-experimental. This PR deprecates `fossa analyze --x-snippet-scan`. Please use `fossa analyze --snippet-scan` instead. ([#1622](https://github.com/fossas/fossa-cli/pull/1622)) diff --git a/docs/README.md b/docs/README.md index c6863642ab..d54c167265 100644 --- a/docs/README.md +++ b/docs/README.md @@ -120,17 +120,6 @@ Concept guides explain the nuances behind how basic FOSSA primitives work. If yo - [Overview and Support Policy](./references/experimental/README.md) -#### Scanning Projects for Source Snippets - -- [Overview](./references/subcommands/snippets.md) -- [Quickstart](./references/subcommands/snippets.md#quickstart) -- [Frequently Asked Questions](./references/subcommands/snippets.md#faq) - -##### Subcommands - -- [`fossa snippets analyze`](./references/subcommands/snippets/analyze.md) -- [`fossa snippets commit`](./references/subcommands/snippets/commit.md) - ### References #### CLI commands @@ -140,7 +129,6 @@ Concept guides explain the nuances behind how basic FOSSA primitives work. If yo - [`fossa init`](./references/subcommands/init.md): Generate sample config files. - [`fossa list-targets`](./references/subcommands/list-targets.md): Retrieve a list of filterable targets in a project. - [`fossa report`](./references/subcommands/report.md): Download a report of the most recent scan of a project. -- [`fossa snippets`](./references/subcommands/snippets.md): Analyze snippets of a project and check if they exist in other open source projects FOSSA knows about. - [`fossa test`](./references/subcommands/test.md): View the results of the most recent scan of a project. - [`fossa release-group`](./references/subcommands/release-group.md): Interact with FOSSA release groups. - [`fossa project`](./references/subcommands/project.md): Interact with FOSSA projects. diff --git a/docs/references/subcommands/snippets.md b/docs/references/subcommands/snippets.md deleted file mode 100644 index 7665c2dd67..0000000000 --- a/docs/references/subcommands/snippets.md +++ /dev/null @@ -1,106 +0,0 @@ -## `fossa snippets` - -This subcommand is the home for FOSSA's snippet scanning feature. - -It is made up of two subcommands: - -- [`fossa snippets analyze`](./snippets/analyze.md) -- [`fossa snippets commit`](./snippets/commit.md) - -See the pages linked above for more details. - -## Quickstart - -```shell -# Set your API key. Get this from the FOSSA web application. -# On Windows, use this instead: $env:FOSSA_API_KEY=XXXX -export FOSSA_API_KEY=XXXX - -# Navigate to your project directory. -cd $MY_PROJECT_DIR - -# Analyze the project for local snippet matches. -# Match data is output to the directory specified to the `-o` or `--output` argument. -# If desired, you can manually review the matches output to the directory. -fossa snippets analyze -o snippets - -# Commit matched snippets to a `fossa-deps` file. -# Provide it the same directory provided to `fossa snippets analyze`. -# This creates a `fossa-deps` file in your project. -# -# Note that you can control what kinds of snippets are committed; -# see subcommand documentation for more details. -fossa snippets commit --analyze-output snippets - -# Run a standard FOSSA analysis, which will also upload snippet scanned dependencies, -# since they were stored in your `fossa-deps` file. -fossa analyze -``` - -## FAQ - -### Is my source code sent to FOSSA's servers? - -**Short version: No.** More detail explaining this is below. - -FOSSA CLI fingerprints your first party source code but does not send it to the server. -The fingerprint is a SHA-256 hashed representation of the content that made up the snippet. - -FOSSA CLI does send the fingerprint to the server, but since SHA-256 hashes are -[cryptographically secure](https://en.wikipedia.org/wiki/SHA-2), it is effectively not possible -for FOSSA to reproduce the original code that went into the snippet. - -Of course, if the fingerprint matches FOSSA could then infer that the project contains that snippet of code, -but since FOSSA CLI does not send any additional context in the file there's no way for FOSSA or anyone else -to make use of this information. - -The code to perform this is open source in this CLI; -users can also utilize tooling such as [echotraffic](https://github.com/fossas/echotraffic) -to report the information being uploaded. - -### How does FOSSA snippet scanning work? - -FOSSA snippet scanning operates over a matrix of options: - -``` -Targets × Kinds × Methods -``` - -Valid options for `Targets` are: - -Target | Description ------------|----------------------------------------------------------------------- -`Function` | Considers function declarations in the source code as snippet targets. - -Valid options for `Kinds` are: - -Kind | Description -------------|---------------------------------------------- -`Full` | The full expression that makes up the target. -`Signature` | The function signature of `Function` targets. -`Body` | The function body of `Function` targets. - -Valid options for `Methods` are: - -Method | Description ---------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------- -`Raw` | The expression that makes up the target as written in the source code file. -`NormalizedSpace` | The expression with any character in the Unicode [whitespace character class][] replaced with a space, and any contiguous spaces collapsed to a single space. -`NormalizedComment` | The expression with comments removed, as defined by the source code language. -`NormalizedCode` | Equivalent to `NormalizedComment` followed by `NormalizedSpace`. - -Given these options, the fully defined matrix of options is as follows: - -``` -{Function} × {Full, Signature, Body} × {Raw, NormalizedSpace, NormalizedComment, NormalizedCode} -``` - -FOSSA then scans open source projects for these snippets and records them along with their metadata, -such as where in the file the snippet originated and from what project. - -Finally, when users scan their first-party projects, FOSSA extracts snippets in the same manner -and compares the fingerprints of the content of those snippets against the database. -If a match is found, FOSSA reports all open source projects in which the snippet was found, -along with recorded metadata about that snippet. - -[whitespace character class]: https://en.wikipedia.org/wiki/Unicode_character_property#Whitespace diff --git a/docs/references/subcommands/snippets/analyze.md b/docs/references/subcommands/snippets/analyze.md deleted file mode 100644 index 472b05554f..0000000000 --- a/docs/references/subcommands/snippets/analyze.md +++ /dev/null @@ -1,110 +0,0 @@ -## `fossa snippets analyze` - -This subcommand extracts snippets from a user project and compares them to the FOSSA database of snippets. -Any matches are then written to the directory provided. - -## Options - -Argument | Required | Default | Description ----------------------|----------|------------------------|-------------------------------------------------------------------------------------------------------------------------------------- -`-o` / `--output` | Yes | None | The directory to which matches are output. -`--debug` | No | No | Enables debug mode. Note that debug bundles are not currently exported with `fossa snippets`, but this output is similarly useful. -`--overwrite-output` | No | No | If specified, overwrites the directory indicated by `--output`. -`--target` | No | `function` | If specified, extracts and matches only the specified targets. Specify multiple options by providing this argument multiple times. -`--kind` | No | `full, snippet, body` | If specified, extracts and matches only the specified kinds. Specify multiple options by providing this argument multiple times. -`--transform` | No | `space, comment, code` | If specified, extracts and matches only the specified transforms. Specify multiple options by providing this argument multiple times. - -> [!NOTE] -> `--transform` corresponds to the `Normalized` methods [listed here](../snippets.md#how-does-fossa-snippet-scanning-work). -> The `Raw` method is always enabled and cannot be disabled. - -## Output - -Matches are written to the location specified by the `--output` (or `-o`) argument. - -The output directory consists of a set of flat files, each representing a file in the scan directory -that had at least one matching snippet. These files are named with the path of the file relative to -the scan directory, with any path separators replaced by underscores, and a `.json` extension appended. - -For example, the following project: -``` -example-project/ - lib/ - lib.c - vendor/ - openssh/ - openssh.c - main.c -``` - -When scanned like `fossa snippets analyze -o snippets`, -would be presented like the below if all files contained a snippet match: -``` -snippets/ - lib_lib.c.json - lib_vendor_openssh_openssh.c.json - main.c.json -``` - -The content of each of these files is a JSON encoded array of matches, -where each object in the array consists of the following keys: - -Key | Description ---------------------|------------------------------------------------------------------------------- -`found_in` | The relative path of the local file in which the snippet match was found. -`local_text` | The text that matched the snippet in the local file. -`local_snippet` | Information about the snippet extracted from the local file. -`matching_snippets` | A collection of snippets from the FOSSA knowledgebase that match this snippet. - -The `local_snippet` object has the following keys: - -Key | Description ---------------|--------------------------------------------------------------------------- -`fingerprint` | The base64 representation of the snippet fingerprint. -`target` | The kind of source code item that matched for this snippet. -`kind` | The kind of snippet that was matched. -`method` | The normalization method used on the matching snippet. -`file_path` | The path of the file containing the snippet, relative to the project root. -`byte_start` | The byte index in the file at which the snippet begins. -`byte_end` | The byte index in the file at which the snippet ends. -`line_start` | The line number in the file at which the snippet begins. -`line_end` | The line number in the file at which the snippet ends. -`col_start` | The column number on the `line_start` at which the snippet begins. -`col_end` | The column number on the `line_end` at which the snippet ends. -`language` | The language of the identified snippet. - -Each entry in the `matching_snippets` collection has the following keys: - -Key | Description ---------------|--------------------------------------------------------------------------- -`locator` | The FOSSA identifier for the project to which this snippet belongs. -`fingerprint` | The base64 representation of the snippet fingerprint. -`target` | The kind of source code item that matched for this snippet. -`kind` | The kind of snippet that was matched. -`method` | The normalization method used on the matching snippet. -`file_path` | The path of the file containing the snippet, relative to the project root. -`byte_start` | The byte index in the file at which the snippet begins. -`byte_end` | The byte index in the file at which the snippet ends. -`line_start` | The line number in the file at which the snippet begins. -`line_end` | The line number in the file at which the snippet ends. -`col_start` | The column number on the `line_start` at which the snippet begins. -`col_end` | The column number on the `line_end` at which the snippet ends. -`language` | The language of the identified snippet. -`ingest_id` | The ingestion run that discovered this snippet (not meaningful to users). - -# Correcting Matches - -In order to correct matches, users may manually edit the contents of this directory -or files within the directory to alter or remove matches. - -For example, if a certain snippet is found in the local code that matches -a snippet in the FOSSA knowledgebase, but it's known to be a false positive, -users can script the removal of that snippet match from this directory prior to -committing these results in a FOSSA scan. - -# Next Steps - -After running `fossa snippets analyze`, the next step is to run `fossa snippets commit`. - -These are separate steps to give users the ability to edit or review the matched data -prior to submitting the results to FOSSA. diff --git a/docs/references/subcommands/snippets/commit.md b/docs/references/subcommands/snippets/commit.md deleted file mode 100644 index 9b038254c3..0000000000 --- a/docs/references/subcommands/snippets/commit.md +++ /dev/null @@ -1,44 +0,0 @@ -## `fossa snippets commit` - -This subcommand commits the analysis performed in the `analyze` subcommand into a `fossa-deps` file ([reference](../../files/fossa-deps.md)). -For more information on possible options, run `fossa snippets commit --help`. - -## Options - -Argument | Required | Default | Description --------------------------|----------|------------------------|----------------------------------------------------------------------------------------------------------------------------------------------- -`--analyze-output` | Yes | None | The directory to which `fossa snippets analyze` output its matches. -`--debug` | No | No | Enables debug mode. Note that debug bundles are not currently exported with `fossa snippets`, but this output is similarly useful. -`--overwrite-fossa-deps` | No | No | If specified, overwrites the `fossa-deps` file if present. -`--target` | No | `function` | If specified, commits matches consisting of only the specified targets. Specify multiple options by providing this argument multiple times. -`--kind` | No | `full, snippet, body` | If specified, commits matches consisting of only the specified kinds. Specify multiple options by providing this argument multiple times. -`--transform` | No | `space, comment, code` | If specified, commits matches consisting of only the specified transforms. Specify multiple options by providing this argument multiple times. -`--format` | No | `yml` | Allows configuring the format of the generated `fossa-deps` file. - -> [!NOTE] -> `--transform` corresponds to the `Normalized` methods [listed here](../snippets.md#how-does-fossa-snippet-scanning-work). -> The `Raw` method is always enabled and cannot be disabled. - -## Input - -The primary thing this subcommand requires is the path to the directory in which the output of `analyze` -was written. Users can also alter which kinds of matches to commit, and customize the output format -of the created `fossa-deps` file. - -## Output - -The result of this subcommand is a `fossa-deps` file written to the root of the project directory. - -> [!NOTE] -> This subcommand will not overwrite an existing `fossa-deps` file by default, -> and currently does not merge its output into an existing `fossa-deps` file. -> -> However, users can customize the output format (via `--format`) and then -> perform scripted merges themselves. - -## Next Steps - -After running `fossa snippets commit`, the next step is to run `fossa analyze` on the project. - -FOSSA CLI will then pick up the dependencies reported in that `fossa-deps` file and report them -as dependencies of the project. diff --git a/extlib/millhone/README.md b/extlib/millhone/README.md index 36dd4f6829..7537afca96 100644 --- a/extlib/millhone/README.md +++ b/extlib/millhone/README.md @@ -1,7 +1,6 @@ # `millhone` -This CLI is used for FOSSA snippet scanning, -and will be accessible via `fossa snippets` in future FOSSA CLI releases. +This CLI is used for FOSSA container JAR analysis. # Subcommands diff --git a/extlib/millhone/docs/subcommands/analyze.md b/extlib/millhone/docs/subcommands/analyze.md index e8662a6888..0eb13f4c36 100644 --- a/extlib/millhone/docs/subcommands/analyze.md +++ b/extlib/millhone/docs/subcommands/analyze.md @@ -22,7 +22,7 @@ example-project/ main.c ``` -When scanned like `fossa snippets analyze -o snippets`, +When scanned like `millhone analyze -o snippets`, would be presented like the below if all files contained a snippet match: ``` snippets/ @@ -89,7 +89,7 @@ committing these results in a FOSSA scan. # Next Steps -After running `fossa snippets analyze`, the next step is to run `fossa snippets commit`. +After running `millhone analyze`, the next step is to run `millhone commit`. These are separate steps to give users the ability to edit or review the matched data prior to submitting the results to FOSSA. diff --git a/spectrometer.cabal b/spectrometer.cabal index 4b45e04ad4..0226640184 100644 --- a/spectrometer.cabal +++ b/spectrometer.cabal @@ -230,7 +230,6 @@ library App.Fossa.Config.SBOM.Analyze App.Fossa.Config.SBOM.Common App.Fossa.Config.SBOM.Test - App.Fossa.Config.Snippets App.Fossa.Config.Test App.Fossa.Container App.Fossa.Container.AnalyzeNative @@ -281,9 +280,6 @@ library App.Fossa.RunThemis App.Fossa.SBOM App.Fossa.SBOM.Analyze - App.Fossa.Snippets - App.Fossa.Snippets.Analyze - App.Fossa.Snippets.Commit App.Fossa.Subcommand App.Fossa.Test App.Fossa.VendoredDependency diff --git a/src/App/Fossa/Config/Snippets.hs b/src/App/Fossa/Config/Snippets.hs deleted file mode 100644 index 44bebae14f..0000000000 --- a/src/App/Fossa/Config/Snippets.hs +++ /dev/null @@ -1,282 +0,0 @@ -module App.Fossa.Config.Snippets ( - mkSubCommand, - SnippetsConfig (..), - SnippetsCommand, - CommitOutputFormat (..), - SnippetKind (..), - SnippetTarget (..), - SnippetTransform (..), - AnalyzeConfig (..), - CommitConfig (..), - labelForKind, - labelForTarget, - labelForTransform, -) where - -import App.Fossa.Config.Common (apiKeyOpt, baseDirArg, collectBaseDir, endpointOpt) -import App.Fossa.Subcommand (EffStack, GetCommonOpts, GetSeverity (..), SubCommand (..)) -import App.Types (BaseDir) -import Control.Carrier.Lift (sendIO) -import Control.Effect.Diagnostics (Diagnostics, Has) -import Control.Effect.Lift (Lift) -import Data.Aeson (ToJSON, defaultOptions, genericToEncoding, toEncoding) -import Data.List qualified as List -import Data.String.Conversion (ToString, ToText, toString, toText) -import Data.Text (Text) -import Effect.Logger (Severity (..)) -import Effect.ReadFS (ReadFS) -import GHC.Generics (Generic) -import Options.Applicative (InfoMod, Parser, command, eitherReader, info, long, many, metavar, option, optional, progDescDoc, short, strOption, subparser, switch, (<|>)) -import Path (Abs, Dir, Path) -import Path.IO qualified as Path -import Style (applyFossaStyle, formatStringToDoc, stringToHelpDoc) -import Text.URI (URI) - -data SnippetsCommand - = CommandAnalyze - FilePath -- Scan root - Bool -- Debug - (Maybe URI) -- The FOSSA endpoint. Not currently used, but accepted for backwards compatibility. - (Maybe Text) -- The FOSSA API key. Not currently used, but accepted for backwards compatibility. - FilePath -- Output directory - Bool -- Whether to overwrite output directory - [SnippetTarget] - [SnippetKind] - [SnippetTransform] - | CommandCommit - FilePath -- Scan root - Bool -- Debug - (Maybe URI) -- The FOSSA endpoint. Not currently used, but accepted for backwards compatibility. - (Maybe Text) -- The FOSSA API key. Not currently used, but accepted for backwards compatibility. - FilePath -- Analyze's output directory - Bool -- Whether to overwrite output file - (Maybe CommitOutputFormat) - [SnippetTarget] - [SnippetKind] - [SnippetTransform] - -snippetsInfo :: InfoMod a -snippetsInfo = progDescDoc $ formatStringToDoc "FOSSA snippet scanning" - -snippetsAnalyzeInfo :: InfoMod a -snippetsAnalyzeInfo = progDescDoc $ formatStringToDoc "Analyze a local project for snippet matches" - -snippetsCommitInfo :: InfoMod a -snippetsCommitInfo = progDescDoc $ formatStringToDoc "Commit matches discovered during analyze into a fossa-deps file" - -instance GetSeverity SnippetsCommand where - getSeverity :: SnippetsCommand -> Severity - getSeverity (CommandAnalyze _ analyzeDebug _ _ _ _ _ _ _) = if analyzeDebug then SevDebug else SevInfo - getSeverity (CommandCommit _ commitDebug _ _ _ _ _ _ _ _) = if commitDebug then SevDebug else SevInfo - -instance GetCommonOpts SnippetsCommand - -mkSubCommand :: (SnippetsConfig -> EffStack ()) -> SubCommand SnippetsCommand SnippetsConfig -mkSubCommand = SubCommand "snippets" snippetsInfo cliParser noLoadConfig mergeOpts - where - noLoadConfig = const $ pure Nothing - -cliParser :: Parser SnippetsCommand -cliParser = analyze <|> commit - where - analyze = subparser . command "analyze" $ info analyzeOpts snippetsAnalyzeInfo - analyzeOpts = - CommandAnalyze - <$> baseDirArg - <*> switch (applyFossaStyle <> long "debug" <> stringToHelpDoc "Enable debug logging") - <*> endpointOpt - <*> apiKeyOpt - <*> strOption (applyFossaStyle <> long "output" <> short 'o' <> stringToHelpDoc "The directory to which matches are output") - <*> switch (applyFossaStyle <> long "overwrite-output" <> stringToHelpDoc "If specified, overwrites the output directory if it exists") - <*> many (option (eitherReader parseTarget) (applyFossaStyle <> long "target" <> stringToHelpDoc "Analyze this combination of targets" <> metavar "TARGET")) - <*> many (option (eitherReader parseKind) (applyFossaStyle <> long "kind" <> stringToHelpDoc "Analyze this combination of kinds" <> metavar "KIND")) - <*> many (option (eitherReader parseTransform) (applyFossaStyle <> long "transform" <> stringToHelpDoc "Analyze this combination of transforms" <> metavar "TRANSFORM")) - commit = subparser . command "commit" $ info commitOpts snippetsCommitInfo - commitOpts = - CommandCommit - <$> baseDirArg - <*> switch (applyFossaStyle <> long "debug" <> stringToHelpDoc "Enable debug logging") - <*> endpointOpt - <*> apiKeyOpt - <*> strOption (applyFossaStyle <> long "analyze-output" <> stringToHelpDoc "The directory to which 'analyze' matches were saved") - <*> switch (applyFossaStyle <> long "overwrite-fossa-deps" <> stringToHelpDoc "If specified, overwrites the 'fossa-deps' file if it exists") - <*> optional (option (eitherReader parseCommitOutputFormat) (applyFossaStyle <> long "format" <> stringToHelpDoc "The output format for the generated `fossa-deps` file" <> metavar "FORMAT")) - <*> many (option (eitherReader parseTarget) (applyFossaStyle <> long "target" <> stringToHelpDoc "Commit this combination of targets" <> metavar "TARGET")) - <*> many (option (eitherReader parseKind) (applyFossaStyle <> long "kind" <> stringToHelpDoc "Commit this combination of kinds" <> metavar "KIND")) - <*> many (option (eitherReader parseTransform) (applyFossaStyle <> long "transform" <> stringToHelpDoc "Commit this combination of transforms" <> metavar "TRANSFORM")) - -mergeOpts :: - ( Has Diagnostics sig m - , Has (Lift IO) sig m - , Has ReadFS sig m - ) => - Maybe FilePath -> - a -> - b -> - SnippetsCommand -> - m SnippetsConfig -mergeOpts _ _ _ (CommandAnalyze path debug _ _ output overwrite targets kinds transforms) = do - root <- collectBaseDir path - output' <- sendIO $ Path.resolveDir' output - pure . Analyze $ AnalyzeConfig root debug output' overwrite targets kinds transforms -mergeOpts _ _ _ (CommandCommit path debug _ _ analyzeOutput overwrite format targets kinds transforms) = do - root <- collectBaseDir path - analyzeOutput' <- sendIO $ Path.resolveDir' analyzeOutput - pure . Commit $ CommitConfig root debug analyzeOutput' overwrite format targets kinds transforms - -data SnippetsConfig - = Analyze AnalyzeConfig - | Commit CommitConfig - deriving (Show, Generic) - -instance ToJSON SnippetsConfig where - toEncoding = genericToEncoding defaultOptions - -data AnalyzeConfig = AnalyzeConfig - { analyzeScanDir :: BaseDir - , analyzeDebug :: Bool - , analyzeOutput :: Path Abs Dir - , analyzeOverwriteOutput :: Bool - , analyzeTargets :: [SnippetTarget] - , analyzeKinds :: [SnippetKind] - , analyzeTransforms :: [SnippetTransform] - } - deriving (Show, Generic) - -instance ToJSON AnalyzeConfig where - toEncoding = genericToEncoding defaultOptions - -data CommitConfig = CommitConfig - { commitScanDir :: BaseDir - , commitDebug :: Bool - , commitAnalyzeOutput :: Path Abs Dir - , commitOverwriteFossaDeps :: Bool - , commitOutputFormat :: Maybe CommitOutputFormat - , commitTargets :: [SnippetTarget] - , commitKinds :: [SnippetKind] - , commitTransforms :: [SnippetTransform] - } - deriving (Show, Generic) - -instance ToJSON CommitConfig where - toEncoding = genericToEncoding defaultOptions - --- | The targets of snippets to extract. --- Reference: @millhone::extract::Target@. -data SnippetTarget - = SnippetTargetFunction - deriving (Eq, Enum, Bounded, Show, Generic) - -instance ToJSON SnippetTarget where - toEncoding = genericToEncoding defaultOptions - -parseTarget :: String -> Either String SnippetTarget -parseTarget input = case List.find (\t -> toString t == input) optionsTarget of - Just found -> Right found - Nothing -> Left $ generateParseError input (toString <$> optionsTarget) - -optionsTarget :: [SnippetTarget] -optionsTarget = enumFromTo minBound maxBound - -instance ToText SnippetTarget where - toText :: SnippetTarget -> Text - toText SnippetTargetFunction = "function" - -instance ToString SnippetTarget where - toString :: SnippetTarget -> String - toString = toString . toText - -labelForTarget :: Text -labelForTarget = "--target" - --- | The kind of item this snippet represents. --- Reference: @millhone::extract::Kind@. -data SnippetKind - = SnippetKindSignature - | SnippetKindBody - | SnippetKindFull - deriving (Eq, Enum, Bounded, Show, Generic) - -instance ToJSON SnippetKind where - toEncoding = genericToEncoding defaultOptions - -instance ToText SnippetKind where - toText :: SnippetKind -> Text - toText SnippetKindSignature = "signature" - toText SnippetKindBody = "body" - toText SnippetKindFull = "full" - -instance ToString SnippetKind where - toString :: SnippetKind -> String - toString = toString . toText - -parseKind :: String -> Either String SnippetKind -parseKind input = case List.find (\t -> toString t == input) optionsKind of - Just found -> Right found - Nothing -> Left $ generateParseError input (toString <$> optionsKind) - -optionsKind :: [SnippetKind] -optionsKind = enumFromTo minBound maxBound - -labelForKind :: Text -labelForKind = "--kind" - --- | The normalization used to extract this snippet. --- Reference: @millhone::extract::Transform@. -data SnippetTransform - = SnippetTransformCode - | SnippetTransformComment - | SnippetTransformSpace - deriving (Eq, Enum, Bounded, Show, Generic) - -instance ToJSON SnippetTransform where - toEncoding = genericToEncoding defaultOptions - -instance ToText SnippetTransform where - toText :: SnippetTransform -> Text - toText SnippetTransformCode = "code" - toText SnippetTransformComment = "comment" - toText SnippetTransformSpace = "space" - -instance ToString SnippetTransform where - toString :: SnippetTransform -> String - toString = toString . toText - -parseTransform :: String -> Either String SnippetTransform -parseTransform input = case List.find (\t -> toString t == input) optionsTransform of - Just found -> Right found - Nothing -> Left $ generateParseError input (toString <$> optionsTransform) - -optionsTransform :: [SnippetTransform] -optionsTransform = enumFromTo minBound maxBound - -labelForTransform :: Text -labelForTransform = "--transform" - -data CommitOutputFormat - = Yml - | Json - deriving (Eq, Enum, Bounded, Show, Generic) - -instance ToJSON CommitOutputFormat where - toEncoding = genericToEncoding defaultOptions - -instance ToText CommitOutputFormat where - toText :: CommitOutputFormat -> Text - toText Yml = "yml" - toText Json = "json" - -instance ToString CommitOutputFormat where - toString :: CommitOutputFormat -> String - toString = toString . toText - -parseCommitOutputFormat :: String -> Either String CommitOutputFormat -parseCommitOutputFormat input = case List.find (\t -> toString t == input) optionsCommitOutputFormat of - Just found -> Right found - Nothing -> Left $ generateParseError input (toString <$> optionsCommitOutputFormat) - -optionsCommitOutputFormat :: [CommitOutputFormat] -optionsCommitOutputFormat = enumFromTo minBound maxBound - -generateParseError :: String -> [String] -> String -generateParseError input options = "'" <> input <> "' is not a valid option; expected one of: " <> List.intercalate ", " options diff --git a/src/App/Fossa/Main.hs b/src/App/Fossa/Main.hs index a480ec959a..4e9e4882cc 100644 --- a/src/App/Fossa/Main.hs +++ b/src/App/Fossa/Main.hs @@ -12,7 +12,6 @@ import App.Fossa.Project qualified as Project import App.Fossa.ReleaseGroup qualified as ReleaseGroup import App.Fossa.Report qualified as Report import App.Fossa.SBOM qualified as SBOM -import App.Fossa.Snippets qualified as Snippets import App.Fossa.Subcommand (GetCommonOpts, GetSeverity, SubCommand (..), runSubCommand) import App.Fossa.Test qualified as Test import App.Fossa.VSI.IAT.AssertUserDefinedBinaries qualified as LinkBins @@ -102,7 +101,6 @@ subcommands = public <|> private , decodeSubCommand Container.containerSubCommand , decodeSubCommand ListTargets.listSubCommand , decodeSubCommand LinkBins.linkBinsSubCommand - , decodeSubCommand Snippets.snippetsSubCommand , decodeSubCommand SBOM.sbomSubCommand , initCommand , feedbackCommand diff --git a/src/App/Fossa/Snippets.hs b/src/App/Fossa/Snippets.hs deleted file mode 100644 index 8d607d6806..0000000000 --- a/src/App/Fossa/Snippets.hs +++ /dev/null @@ -1,46 +0,0 @@ --- Types in this module are tightly based on the types in the Millhone CLI. --- --- In the documentation for this module, Millhone symbols are written --- using Rust-style paths (e.g. @millhone::extract::Target@). --- --- The Millhone CLI is at @extlib/millhone@. --- --- Notable exceptions: --- - Auth config is not included here. --- The plan is to use FOSSA reverse proxying eventually, --- and until that's done Millhone CLI hard codes authentication information. --- - Logging config is not included here. --- Instead FOSSA CLI automatically configures it based on --- its configured log severity. - -module App.Fossa.Snippets ( - snippetsMain, - snippetsSubCommand, -) where - -import App.Fossa.Config.Snippets (SnippetsCommand, SnippetsConfig (..), mkSubCommand) -import App.Fossa.Snippets.Analyze (analyzeWithMillhone) -import App.Fossa.Snippets.Commit (commitWithMillhone) -import App.Fossa.Subcommand (SubCommand) -import Control.Algebra (Has) -import Control.Effect.Diagnostics (Diagnostics) -import Control.Effect.Lift (Lift) -import Effect.Exec (Exec) -import Effect.Logger (Logger, logInfo) - -snippetsSubCommand :: SubCommand SnippetsCommand SnippetsConfig -snippetsSubCommand = mkSubCommand snippetsMain - -snippetsMain :: - ( Has (Lift IO) sig m - , Has Exec sig m - , Has Diagnostics sig m - , Has Logger sig m - ) => - SnippetsConfig -> - m () -snippetsMain subcommand = do - logInfo "Running FOSSA snippets" - case subcommand of - Analyze cfg -> analyzeWithMillhone cfg - Commit cfg -> commitWithMillhone cfg diff --git a/src/App/Fossa/Snippets/Analyze.hs b/src/App/Fossa/Snippets/Analyze.hs deleted file mode 100644 index 580623711c..0000000000 --- a/src/App/Fossa/Snippets/Analyze.hs +++ /dev/null @@ -1,45 +0,0 @@ -{-# LANGUAGE RecordWildCards #-} - -module App.Fossa.Snippets.Analyze ( - analyzeWithMillhone, -) where - -import App.Fossa.Config.Snippets (AnalyzeConfig (..), labelForKind, labelForTarget, labelForTransform) -import App.Fossa.EmbeddedBinary (BinaryPaths, toPath, withMillhoneBinary) -import App.Types (BaseDir (unBaseDir)) -import Control.Algebra (Has) -import Control.Effect.Diagnostics (Diagnostics) -import Control.Effect.Lift (Lift) -import Data.String.Conversion (toText) -import Effect.Exec (AllowErr (Never), Command (..), Exec, argFromPath, argsLabeled, execEffectful) -import Effect.Logger (Logger) -import Path (Abs, Dir, Path) - -analyzeWithMillhone :: - ( Has (Lift IO) sig m - , Has Exec sig m - , Has Diagnostics sig m - , Has Logger sig m - ) => - AnalyzeConfig -> - m () -analyzeWithMillhone conf = withMillhoneBinary $ \bin -> execEffectful root $ mkCmd bin root conf - where - root = unBaseDir $ analyzeScanDir conf - -mkCmd :: BinaryPaths -> Path Abs Dir -> AnalyzeConfig -> Command -mkCmd bin root AnalyzeConfig{..} = - Command - { cmdName = toText $ toPath bin - , cmdArgs = concat [debug, cmd, output, overwriteOutput, targets, kinds, transforms, dir] - , cmdAllowErr = Never - } - where - cmd = ["analyze"] - dir = [argFromPath root] - debug = if analyzeDebug then ["--log-level", "debug", "--log-format", "json"] else [] - targets = if null analyzeTargets then [] else argsLabeled labelForTarget analyzeTargets - kinds = if null analyzeKinds then [] else argsLabeled labelForKind analyzeKinds - transforms = if null analyzeTransforms then [] else argsLabeled labelForTransform analyzeTransforms - output = ["--output", argFromPath analyzeOutput] - overwriteOutput = if analyzeOverwriteOutput then ["--overwrite-output"] else [] diff --git a/src/App/Fossa/Snippets/Commit.hs b/src/App/Fossa/Snippets/Commit.hs deleted file mode 100644 index dbaa3ddaf0..0000000000 --- a/src/App/Fossa/Snippets/Commit.hs +++ /dev/null @@ -1,48 +0,0 @@ -{-# LANGUAGE RecordWildCards #-} - -module App.Fossa.Snippets.Commit ( - commitWithMillhone, -) where - -import App.Fossa.Config.Snippets (CommitConfig (..), labelForKind, labelForTarget, labelForTransform) -import App.Fossa.EmbeddedBinary (BinaryPaths, toPath, withMillhoneBinary) -import App.Types (unBaseDir) -import Control.Algebra (Has) -import Control.Effect.Diagnostics (Diagnostics) -import Control.Effect.Lift (Lift) -import Data.String.Conversion (toText) -import Effect.Exec (AllowErr (Never), Command (..), Exec, argFromPath, argsLabeled, execEffectful) -import Effect.Logger (Logger) -import Path (Abs, Dir, Path) - -commitWithMillhone :: - ( Has (Lift IO) sig m - , Has Exec sig m - , Has Diagnostics sig m - , Has Logger sig m - ) => - CommitConfig -> - m () -commitWithMillhone conf = withMillhoneBinary $ \bin -> execEffectful root $ mkCmd bin root conf - where - root = unBaseDir $ commitScanDir conf - -mkCmd :: BinaryPaths -> Path Abs Dir -> CommitConfig -> Command -mkCmd bin root CommitConfig{..} = - Command - { cmdName = toText $ toPath bin - , cmdArgs = concat [debug, cmd, output, format, overwriteOutput, targets, kinds, transforms, dir] - , cmdAllowErr = Never - } - where - cmd = ["commit"] - dir = [argFromPath root] - debug = if commitDebug then ["--log-level", "debug", "--log-format", "json"] else [] - targets = if null commitTargets then [] else argsLabeled labelForTarget commitTargets - kinds = if null commitKinds then [] else argsLabeled labelForKind commitKinds - transforms = if null commitTransforms then [] else argsLabeled labelForTransform commitTransforms - output = ["--analyze-output-dir", argFromPath commitAnalyzeOutput] - format = case commitOutputFormat of - Just format' -> ["--format", toText format'] - Nothing -> [] - overwriteOutput = if commitOverwriteFossaDeps then ["--overwrite-fossa-deps"] else []