Does not properly segment within quotations

[Hgherzog]

When dealing with a long statement of facts quoted from legal text, the text is not split up within left double quotations and write double quotations. this is different than the " characterI cannot share the text here as it deals with sensitive content.

import pysbd
seg = pysbd.Segmenter(language='en')
sentences = seg.segment(above_text)

Returns a lot of length 1 and does not split by sentences. The expected behavior is to split up into sentences within the quotations.

[libTorrentUser]

I confirm the... bug? Not sure if it is a bug or intentional but

import pysbd

text = 'He said "hello. And then world."'
seg = pysbd.segmenter.Segmenter(language='en', clean=True)
print(seg.segment(text))

['He said "hello. And then world."']

I was expecting

[
    'He said', 
    '"hello.',
    'And then world."'
]

It almost does the right thing when using single quotes. Almost. The sentence is split correctly, but it considers the terminating single quote as a sentence

import pysbd

text = "He said 'hello. And then world.'"
seg = pysbd.segmenter.Segmenter(language='en', clean=True)
print(seg.segment(text))

[
    "He said 'hello.",
    'And then world.',
    "'"]

[ghnp5]

diasks2/pragmatic_segmenter#13 and #45 seems to have the answer - it was a design choice.

But I wonder if we could have an option to change that behavior, as a parameter in the constructor... ?

[joprice]

Not sure if this is related, but a text like

Il répond : Oui, je veux bien. Nous pouvons cuisiner quelque chose ensemble !" Ça va être amusant.

results in a single span:

[TextSpan(sent='Il répond : "Oui, je veux bien. Nous pouvons cuisiner quelque chose ensemble !" Ça va être amusant.', start=0, end=99)] [0]

whereas I would expect two - Il répond... and Ça va...