pep8 in readme

Author: kilianFatras

README.md

@@ -23,14 +23,13 @@
 
 ## Description
 
-Conditional Flow Matching (CFM) is a fast way to train continuous normalizing flow (CNF) models. CFM is a simulation-free training objective for continuous normalizing flows that allows conditional generative modeling and speeds up training and inference. CFM's performance closes the gap between CNFs and diffusion models. To spread its use within the machine learning community, we have built a library focused on Flow Matching methods: TorchCFM. TorchCFM is a library showing how Flow Matching methods can be trained and use to deal with image generation, single-cell dynamics and (soon) SO(3) data and tabular data. 
+Conditional Flow Matching (CFM) is a fast way to train continuous normalizing flow (CNF) models. CFM is a simulation-free training objective for continuous normalizing flows that allows conditional generative modeling and speeds up training and inference. CFM's performance closes the gap between CNFs and diffusion models. To spread its use within the machine learning community, we have built a library focused on Flow Matching methods: TorchCFM. TorchCFM is a library showing how Flow Matching methods can be trained and use to deal with image generation, single-cell dynamics and (soon) SO(3) data and tabular data.
 
 <p align="center">
 <img src="assets/169_generated_samples_otcfm.gif" width="600"/>
 <img src="assets/8gaussians-to-moons.gif" />
 </p>
 
-
 The density, vector field, and trajectories of simulation-free CNF training schemes: mapping 8 Gaussians to two moons (above) and a single Gaussian to two moons (below). Action matching with the same architecture (3x64 MLP with SeLU activations) underfits with the ReLU, SiLU, and SiLU activations as suggested in the [example code](https://github.com/necludov/jam), but it seems to fit better under our training setup (Action-Matching (Swish)).
 
 The models to produce the GIFs are stored in `examples/models` and can be visualized with this notebook: [![notebook](https://img.shields.io/static/v1?label=Run%20in&message=Google%20Colab&color=orange&logo=Google%20Cloud)](https://colab.research.google.com/github/atong01/conditional-flow-matching/blob/master/examples/notebooks/model-comparison-plotting.ipynb).
@@ -43,11 +42,12 @@ In our version 1 update we have extracted implementations of the relevant flow m
 
 - `ConditionalFlowMatcher`: $z = (x_0, x_1)$, $q(z) = q(x_0) q(x_1)$
 - `ExactOptimalTransportConditionalFlowMatcher`: $z = (x_0, x_1)$, $q(z) = \\pi(x_0, x_1)$ where $\\pi$ is an exact optimal transport joint. This is used in \[Tong et al. 2023a\] and \[Poolidan et al. 2023\] as "OT-CFM" and "Multisample FM with Batch OT" respectively.
-- `TargetConditionalFlowMatcher`: $z = x_1$, $q(z) = q(x_1)$ as defined in Lipman et al. 2023, learns a flow from a standard normal Gaussian to data using conditional flows which optimally transport the Gaussian to the datapoint (Note that this does not result in the marginal flow being optimal transport.
+- `TargetConditionalFlowMatcher`: $z = x_1$, $q(z) = q(x_1)$ as defined in Lipman et al. 2023, learns a flow from a standard normal Gaussian to data using conditional flows which optimally transport the Gaussian to the datapoint (Note that this does not result in the marginal flow being optimal transport).
 - `SchrodingerBridgeConditionalFlowMatcher`: $z = (x_0, x_1)$, $q(z) = \\pi\_\\epsilon(x_0, x_1)$ where $\\pi\_\\epsilon$ is a an entropically regularized OT plan, although in practice this is often approximated by a minibatch OT plan (See Tong et al. 2023b). The flow-matching variant of this where the marginals are equivalent to the Schrodinger Bridge marginals is known as `SB-CFM` \[Tong et al. 2023a\]. When the score is also known and the bridge is stochastic is called \[SF\]2M \[Tong et al. 2023b\]
 - `VariancePreservingConditionalFlowMatcher`: $z = (x_0, x_1)$ $q(z) = q(x_0) q(x_1)$ but with conditional Gaussian probability paths which preserve variance over time using a trigonometric interpolation as presented in \[Albergo et al. 2023a\].
 
 ## How to cite
+
 This repository contains the code to reproduce the main experiments and illustrations of two preprints:
 
 - [Improving and generalizing flow-based generative models with minibatch optimal transport](https://arxiv.org/abs/2302.00482). We introduce **Optimal Transport Conditional Flow Matching** (OT-CFM), a CFM variant that approximates the dynamical formulation of optimal transport (OT). Based on OT theory, OT-CFM leverages the static optimal transport plan as well as the optimal probability paths and vector fields to approximate dynamic OT.
@@ -87,8 +87,8 @@ A. Tong, N. Malkin, K. Fatras, L. Atanackovic, Y. Zhang, G. Huguet, G. Wolf, Y.
 
 </details>
 
-
 ## V0 -> V1
+
 Major Changes:
 
 - __Added cifar10 examples with an FID of 3.5__
@@ -103,6 +103,7 @@ Major Changes:
 List of implemented papers:
 
 - Flow Matching for Generative Modeling (Lipman et al. 2023) [Paper](https://openreview.net/forum?id=PqvMRDCJT9t)
+- Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow (Liu et al. 2023) [Paper](https://openreview.net/forum?id=XVjTT1nw5z) [Code](https://github.com/gnobitab/RectifiedFlow.git)
 - Building Normalizing Flows with Stochastic Interpolants (Albergo et al. 2023a) [Paper](https://openreview.net/forum?id=li7qeBbCR1t)
 - Action Matching: Learning Stochastic Dynamics From Samples (Neklyudov et al. 2022) [Paper](https://arxiv.org/abs/2210.06662) [Code](https://github.com/necludov/jam)
 - Concurrent work to our OT-CFM method: Multisample Flow Matching: Straightening Flows with Minibatch Couplings (Pooladian et al. 2023) [Paper](https://arxiv.org/abs/2304.14772)
@@ -132,6 +133,7 @@ pip install -r requirements.txt
 ```
 
 To run our jupyter notebooks, use the following commands after installing our package.
+
 ```bash
 # install ipykernel
 conda install -c anaconda ipykernel
@@ -172,8 +174,9 @@ The directory structure of new project looks like this:
 ## ❤️  Code Contributions
 
 This toolbox has been created and is maintained by
-* [Alexander Tong](http://alextong.net)
-* [Kilian Fatras](http://kilianfatras.github.io)
+
+- [Alexander Tong](http://alextong.net)
+- [Kilian Fatras](http://kilianfatras.github.io)
 
 It was initiated from a larger private codebase which loses the original commit history which contains work from other authors of the papers.
 

examples/cifar10/README.md

@@ -9,16 +9,19 @@ This repository is used to reproduce the CIFAR-10 experiments from [1](https://a
 To reproduce the experiments and save the weights, install the requirements from the main repository and then run (runs on a single RTX 2080 GPU):
 
 - For the OT-Conditional Flow Matching method:
+
 ```bash
 python3 train_cifar10.py --model "otcfm" --lr 2e-4 --ema_decay 0.9999 --batch_size 128 --total_steps 400001 --save_step 20000
 ```
 
 - For the Conditional Flow Matching method:
+
 ```bash
 python3 train_cifar10.py --model "cfm" --lr 2e-4 --ema_decay 0.9999 --batch_size 128 --total_steps 400001 --save_step 20000
 ```
 
 - For the original Flow Matching method:
+
 ```bash
 python3 train_cifar10.py --model "fm" --lr 2e-4 --ema_decay 0.9999 --batch_size 128 --total_steps 400001 --save_step 20000
 ```
@@ -29,11 +32,7 @@ To compute the FID from the OT-CFM model at end of training, run:
 python3 compute_fid.py --model "otcfm" --step 400000 --integration_method dopri5
 ```
 
-For the other models, change the "otcfm" argument by "cfm" or "fm". For easy reproducibility of our results, we provide the saved weights of our different methods trained for 400000 iterations. 
-
-- [Trained OT-CFM weights](http://alextong.net)
-- [Trained CFM weights](http://alextong.net)
-- [Trained FM weights](http://alextong.net)
+For the other models, change the "otcfm" argument by "cfm" or "fm". For easy reproducibility of our results, we will provide soon the saved weights of our different methods trained for 400000 iterations.
 
 To recompute the FID, change the PATH variable with where you have saved the downloaded weights.
 

runner/README.md

@@ -26,12 +26,10 @@ python src/train.py trainer.max_epochs=20 datamodule.batch_size=64
 
 You can also train a large set of models in parallel with SLURM as shown in `scripts/two-dim-cfm.sh` which trains the models used in the first 3 lines of Table 2.
 
-
 ## Code Contributions
 
 This repo is extracted from a larger private codebase which loses the original commit history which contains work from other authors of the papers.
 
-
 ## Project Structure
 
 The directory structure of new project looks like this:
@@ -392,4 +390,4 @@ ValueError: Specify tags before launching a multirun!
 
 </details>
 
-<br>
+<br>