# Join Lines (considering - at the end of lines)

Normally when a line ends with -, it means it should be joined differently.

For example, if we join lines of:

The causal part of the competency definition is to distinguish between
those many characteristics that may be studied and measured about a per-
son and those aspects that actually do provide a link to relevant behaviour.
Thus having acquired a particular academic qualification may or may not
be correlated with the capacity to perform a particular job. The qualifica-
tion is scarcely likely – talisman-like – to cause the capacity for perform-
ance. However, a tendency to grasp complexity and to learn new facts and
figures may well be part of the causal chain, and the competency would be
expressed in these terms, not in terms of the possibly related qualification.


It should look like:

The causal part of the competency definition is to distinguish between those many characteristics that may be studied and measured about a person and those aspects that actually do provide a link to relevant behaviour. Thus having acquired a particular academic qualification may or may not be correlated with the capacity to perform a particular job. The qualification is scarcely likely – talisman-like – to cause the capacity for performance. However, a tendency to grasp complexity and to learn new facts and figures may well be part of the causal chain, and the competency would be expressed in these terms, not in terms of the possibly related qualification.

The causal part of the competency definition is to distinguish between those many characteristics that may be studied and measured about a per- son and those aspects that actually do provide a link to relevant behaviour. Thus having acquired a particular academic qualification may or may not be correlated with the capacity to perform a particular job. The qualifica- tion is scarcely likely – talisman-like – to cause the capacity for perform- ance. However, a tendency to grasp complexity and to learn new facts and figures may well be part of the causal chain, and the competency would be expressed in these terms, not in terms of the possibly related qualification.

Please look at the difference between person and per- son, qualification and qualifica- tion.

To solve this problem, I have written a small command line application.

#!/usr/bin/env python3

import sys
import argparse

parser = argparse.ArgumentParser()
args = parser.parse_args()

# taking input from stdin which is empty, so there is neither a stdin nor a file as argument
if sys.stdin.isatty() and args.infile.name == "<stdin>":

paragraph = []

for line in args.infile:
if line.endswith("-\n"):
paragraph.append(line.rstrip('-\n'))
else:
paragraph.append(line.replace("\n", " "))

print(''.join(paragraph))


Please give me some feedback so that I can better this program.

• I think a more precise definition of the problem statement would be helpful for readers and reviewers, so they know up-front what edge cases may or may not need to be handled. For example, if we have two adjacent lines ... scarcely likely – and talisman-like ..., what is the correct output in this case? How about the lines ... scarcely likely – talisman- and like ...? Oct 12 at 4:42
• @Setris If there is a space before - then it will only replace newline with space (so, if we have two adjacent lines ... scarcely likely - and talisman-like ..., it will end up being ... scarcely likely - talisman-like ...). However, I have no idea, how to handle ... scarcely likely – talisman- and like .... I think it will need nlp. Oct 12 at 5:46
• I think for your second use case the lines ... scarcely likely – talisman- and like ..., if we find that the word before the - and the first word of the next line create a valid word (with the dash), in this case talisman-like, we just remove the newline using line.rstrip('\n'). For this stackoverflow.com/questions/3788870/… can be helpful. Oct 12 at 5:52
• Start by mentioning whether your program currently produces the output you want. Oct 12 at 20:39
• @ZacharyVance yes. my program does what I want to do. I was just discussing with Setris Oct 13 at 3:45

### Low hanging fruits

• use a if_main guard in your code. This means you can import it without the whole thing running each time you load it.

• Adding functions is always a good thing.

• Adding typing hints is always nice.

• I have no idea what this part does

# taking input from stdin which is empty, so there is neither a stdin nor a file as argument
if sys.stdin.isatty() and args.infile.name == "<stdin>":


so I removed it. Code seems to be running just as fine.

• Instead I added a required=True keyword to argparse.

With these simple changes the code now looks like

def content_2_proper_paragraph(content:str) -> str:

paragraph = []

for line in content:
if line.endswith("-\n"):
paragraph.append(line.rstrip("-\n"))
else:
paragraph.append(line.replace("\n", " "))

return "".join(paragraph)

if __name__ == "__main__":
import argparse

parser = argparse.ArgumentParser()
"-f", "--file", nargs="?", type=argparse.FileType("r"), required=True
)
args = parser.parse_args()

paragraph = content_2_proper_paragraph(args.file)
print(paragraph)


### Enchantments

So let us keep working on the code. There is still the issue with leading spaces, and some semantics.

• The trailing - to split a word over multiple lines is called a hyphen and so our code should call it that.

• This part screams for a re-factorization

  for line in content:
if line.endswith("-\n"):
paragraph.append(line.rstrip("-\n"))
else:
paragraph.append(line.replace("\n", " "))


If we reorder the logic and start with the paragraph, it looks like this

    for line in content:
paragraph.append(line.rstrip("-\n") if line.endswith("-\n") else line.replace("\n", " "))


The pattern to look out for here is

for x in X:
f(x)


This can be rewritten as map(f, X) or inline [f(x) for x in X]. This means the whole append part should be it's own function. Something like

def content_without_hyphens(
content: str, hyphen: str = "-", keep_if_space: bool = True
):
def remove_trailing_hyphen(line):
*chars, penultimate_char, last_char, _ = line
last_char_is_hyphen = (last_char == hyphen) and (
keep_if_space or penultimate_char != " "
)

return (
"".join(chars)
+ penultimate_char
+ ("" if last_char_is_hyphen else last_char)
)

paragraph = "".join(map(remove_trailing_hyphen, content))
return paragraph


works, but is really messy. I tried to implement a better method to remove the trailing -, but ultimately this is a big fail. Your method with string replace is much cleaner. However, we can not use it directly because we need to use negative lookbehind to figure out if a SPACE preceeds the HYPHEN. The whole code then looks like this

import re
from typing import Annotated

Paragraph = Annotated[str, "a series of sentences"]
Hyphen = Annotated[
str, "Divides long words between the end of one line and the beginning of the next"
]

def paragraph_without_hyphens(paragraph: Paragraph, hyphen: Hyphen = "-") -> Paragraph:

TRAILING_HYPHEN = re.compile(fr"(?<!( )){hyphen}\$")

def remove_trailing_hyphen_regex(line):
return TRAILING_HYPHEN.sub("", line).rstrip("\r\n")

return "".join(map(remove_trailing_hyphen_regex, paragraph))

if __name__ == "__main__":
import argparse

parser = argparse.ArgumentParser()
"-f", "--file", nargs="?", type=argparse.FileType("r"), required=True
)

paragraph = parser.parse_args().file
print(paragraph_without_hyphens(paragraph))

• Actually my program takes input from stdin or file. if sys.stdin.isatty() and args.infile.name == "<stdin>": is checking if there is input from stdin or file. If it has no input from either stdin or file then it can not process anything and gives a message that please give a input. Oct 18 at 4:03