How can one remove excess carriage returns?

[Toxaris]

Originally Posted by dgatwood

Equally off-topic reply: These days, most subtitling is done automatically by speech recognition software. That's why it is so bad. Just imagine saying, "Siri, type 'It was the best of times, it was the worst of times'" and you'll get the idea.

A lot of series aren't broadcasted here (not that is a hinderance...), so that means that subtitles are translated. Some are good, others are on the level of Google Translate. Those are hilarious sometimes... I am glad I don't need them.

[arjaybe]

I use subtitles a lot, especially with the new shows where they think they have to play music continuously, and as loudly as the dialogue.

As for the carriage returns, undoubtedly it will involve a lot of manual proofing.

[Hitch]

Originally Posted by arjaybe

I use subtitles a lot, especially with the new shows where they think they have to play music continuously, and as loudly as the dialogue.

As for the carriage returns, undoubtedly it will involve a lot of manual proofing.

As much as my nostalgia, etc., prompts me to watch "Inspector Lewis," I shall have a party and proffer champagne on the day that Barrington Pheloung retires, and I never have to listen to him overwhelm Masterpiece/Mystery again. Honestly, I think that he's deaf, and so is their mixer. Appalling.

Now, having said that, yes, I do know that the UK systems and ours, for broadcast, are different, and it's highly likely that it sounds just spiffy there. Well, kids, this ain't the 1950's, and any moron can remix something today, for an international audience. I mean, hell, PBS feels justified in "cutting out" the BAD WORDS (Like, ASS. Really???), and editing it for "time," so that we can scrunch our foreheads and wonder about some plot point that was omitted...and they can't remix the bloody thing, to make the music not absurdly over-loud?

(FWIW, I've basically moved over to watching UK and AUS, etc., TV via streaming. Roku, using either Acorn or Netflix or Hulu. At least I don't get my storyline "pre-digested" for me by the bunch of easily-shocked 5th-grade teachers that are doing that at PBS. I confess I haven't tried IL over there, to see if the music is any less onerous, but I'll see if I can spot it and give it a go.)

Hitch

[arjaybe]

Originally Posted by Hitch

(FWIW, I've basically moved over to watching UK and AUS, etc., TV via streaming. Roku, using either Acorn or Netflix or Hulu. At least I don't get my storyline "pre-digested" for me by the bunch of easily-shocked 5th-grade teachers that are doing that at PBS. I confess I haven't tried IL over there, to see if the music is any less onerous, but I'll see if I can spot it and give it a go.)

Hitch

Being in Canada, I get to see the difference between how the same series are aired here and down there, and your guys do seem to be a little more prudish. I guess we have essentially the same broadcast systems though, so I haven't noticed any differences in the audio. I'll try to pay more attention.

[Hitch]

Originally Posted by arjaybe

Being in Canada, I get to see the difference between how the same series are aired here and down there, and your guys do seem to be a little more prudish. I guess we have essentially the same broadcast systems though, so I haven't noticed any differences in the audio. I'll try to pay more attention.

It's annoying as hell. I mean, fine, PBS, I get it, you want to keep your access and all that good crap. So then, why NOT offer the "unedited" versions at 2:00 a.m., or something, so that those of us who don't want to miss plot points don't? Is that too much to ask? I forget which show it was, in the past...year or two? But something happened, that made NO sense. I even thought, "oh, I must not have been paying attention," (as I'm frequently also reading at the same time). When I went back and watched it? Ixnay. They'd chopped out the plot point, which resolved something. I was not a happy camper.

The airwaves are filled with the "lesser" cursing. All the cable networks, even the bloody Sci-Fi Channel and TNT, which is smack dab in the Bible Belt. If those good folks can take it, surely, the rest of us can.

OH, and the butt-erasing? STOP the butt erasing, PLEASE. One more blurred butt will no doubt send me over the edge. Who hasn't seen an ass in their time?

</grump>

Hitch

[Tex2002ans]

Every time this gets brought up, I always bust out the same few Regex:

Regex #1

I use this one to catch a hyphen at the end of a paragraph:

Search: -</p>\s+<p>
Replace:

Never do a "Replace All". Look at each of these manually and decide if it is actually a hard hyphen or a soft hyphen.

<p>"This is an example of wrong non-</p>

<p>alignment."</p>

<p>This is an example of estab-</p>

<p>lishing the sentence.</p>

Regex #2

This one catches any paragraph that ends in a character NOT in the RED part.

In this case, any paragraph that ends WITHOUT a "more than sign" (most likely HTML tags), a "right double quote", a "quotation mark", an "exclamation point", or a "period". (These are valid paragraph endings for the most part).

Search: ([^>”\?\!\.])</p>\s+<p>
Replace: \1

Note: (Make sure there is a space after "\1 ").

<p>This is an example</p>

<p>sentence.</p>

<p>“This is an example of</p>

dialogue.”</p>

<p>This is a long, very long,</p>

<p>very very long example.</p>

<p>And this is an example of one of Tex's (amazing)</p>

<p>Side Notes.</p>

Side Note: Colons are a different beast. Depending on the book, they can be valid paragraph endings or not. Create a separate Search to handle those individually.

Regex #3

Typically a paragraph does not belong with a lowercase letter... so this one catches most stragglers.

Search: <p>[a-z]

<p>He picked three out of the hat:</p>

<p>one, two, three.</p>

Regex #4

This one typically occurs with dialogue... a quotation mark that gets split.

Search: ,”</p>

<p>“Hey! Get back over here,”</p>

<p>Tex said. “You are a buffoon.”</p>

Those Regex should cover most cases. The rest has to be done by thorough manual checking.

Originally Posted by Hitch

BUT, I gotta comment on this.

I have wondered if there's some way for me to donate my time for subtitling. Mr. Hitch needs the subtitles for all the UK and Aussie stuff that we watch, and the subtitles/closed captioning are simply DREADFUL. I mean, dreadful. I don't know how the hell anyone can manage, if they can't fill in the blanks through hearing. It's unbelievable.

It's not any better for US TV; it's pretty much as awful/worse. I know that some of the services are PAID, so that's the worst part. If, like DP and PG, it was all donated time, okay...I could wince and ignore it, but a commercial service? Appalling.

I remember running across an article talking about this:

https://www.viki.com/

Or I remember running across volunteer subtitling for Youtube videos:

https://amara.org/en/

I swear I ran across another one that was similar to Distributed Proofreaders... but I can't seem to find the article/interview in a quick search. I remember the woman started it because her husband was deaf, and the CC + automated subtitles were complete crap. I can't remember for the life of me where I even read the article though. Maybe it was years ago when I was looking into Transcription.

Side Note: Or you could always join the "dark side" and join the Fansubbing community.

Originally Posted by Hitch

It's annoying as hell. I mean, fine, PBS, I get it, you want to keep your access and all that good crap. So then, why NOT offer the "unedited" versions at 2:00 a.m., or something, so that those of us who don't want to miss plot points don't? Is that too much to ask? I forget which show it was, in the past...year or two? But something happened, that made NO sense. I even thought, "oh, I must not have been paying attention," (as I'm frequently also reading at the same time). When I went back and watched it? Ixnay. They'd chopped out the plot point, which resolved something. I was not a happy camper.

Reminds me of them cutting minutes off of shows... and to top it off, speeding the episodes up by a few percent:

https://www.techdirt.com/articles/20...ery-hour.shtml

Stuff like this is partially why I don't watch TV any more... it is absolutely unbearable.

[arjaybe]

Originally Posted by Hitch

OH, and the butt-erasing? STOP the butt erasing, PLEASE. One more blurred butt will no doubt send me over the edge. Who hasn't seen an ass in their time?

</grump>

Hitch

Yeah. That's one of the ways I'm reminded of the source of the episode I'm watching. If the breasts are blurred on the body on the slab, chances are it's PBS.-) Who do they hire to do the blurring? I wonder how the job interview went.

[Doitsu]

Originally Posted by arjaybe

Who do they hire to do the blurring?

The "Blur Man Group."

[theducks]

Originally Posted by Hitch

It's annoying as hell. I mean, fine, PBS, I get it, you want to keep your access and all that good crap. So then, why NOT offer the "unedited" versions at 2:00 a.m., or something, so that those of us who don't want to miss plot points don't? Is that too much to ask? I forget which show it was, in the past...year or two? But something happened, that made NO sense. I even thought, "oh, I must not have been paying attention," (as I'm frequently also reading at the same time). When I went back and watched it? Ixnay. They'd chopped out the plot point, which resolved something. I was not a happy camper.

</grump>

Hitch

I get 3 OTA PBS Stations here (9,22,54).
This is the 21st C. We have DTV. They all have Sub channels.

Frequently they run 4:3 on one, while running 16:9 on the Main (usually the dot 1 ). They could run the untouched version on a sub channel so parents could filter.

BTW My pet peeve: "Reformatted to fit your (4:3) screen" That destroys many wide 'effects' the director was presenting.

[Hitch]

Exactly. It's SUCH Bs.

Between fuzzy asses on actresses and the occasional male (DOUBLE-STANDARD bill****, mind you), the fuzzy boobs, the bleeped or omitted words, and the content clipping/speeding...why on earth should they expect my support?

I used to donate to PBS pretty religiously (yes, that pun was intentional....really, it's irony, but..). I've stopped doing so. Largely for all the reasons mentioned, but also, I feel increasingly that if they are getting the airwaves for the PUBLIC GOOD, they should stop being so adamantly left-wing.

n.b.: both of the rants that are below, mostly belong in the P&R thread. They're not meant to be about that, but someone will, possibly, get their knickers in a twist about it. The first is about public airwaves; the latter is about the increasing encroachment on the idea of intellectual rigor, in determining our OWN choices. However, both are--not deliberately--about liberals in media and other positions. IF someone here gets wonk about it, I'm happy to delete them. I genuinely don't mean to be discussing politics, as opposed to something else, but...you know how folks are. The political bent is, truly, an accidental part--the side-effect--of what's occurring. I would say the same thing if the slant were in any other direction. It's...it's coincidence, more than anything else, but the ACTS involved are not. And it is they about which I am kvetching.

rant about that part:

Spoiler:

I get that they are all, by nature, do-gooders, and like so many journalists, they end up being left-wing. Fine. But here's the thing--unlike CNN, which bids for its airwaves and bandwidth, and all that, THEY, PBS, don't have to. They are here to serve the public good--and half the public aren't Clintonites and whatever all else. I increasingly despair that there will ever be reliable news, without a slant, in this world. There's simply nary one decent news station that can be relied upon for actual NEWS, as opposed to how they predigest it and regurgitate it with their spin. CNN is particularly egregious, but not alone. And no, I'm not just picking on liberals, but it's also true that they hold probably 95% of the so-called "news" stations, channels, etc., in the US. Bloody annoying.

In that vein, about GOOGLE:

Spoiler:

Don't get me started about how shocked I was when I conducted search TESTS on Google, versus Bing, DuckDuckGo, etc. If you try to do something simple--say, search on a phrase like "bad things about Hillary Clinton," or "30 bad things...etc." you will get surprisingly few results. That's what made me go "hmmmm." (No, I wasn't searching for that specific thing; I was trying to research one of the assertions in Clinton Cash, which is how I discovered this.) Then go do the SAME search on Bing or DDG or any other search engine, and be prepared to be surprised. I knew that Google had a conflict of interest in how they direct traffic, of course--website A has Google adverts, versus website B that does not--but this? This is shockingly flagrant. NOT delivering opinions or news, about the liberal candidate, that's "bad?" That is appalling. I've lost a lot of respect for Google, which I thought would at LEAST have the decency to be honest and forthright about SEARCH in the news category. They are, literally, forming public opinion by the simple expedient of FILTERING out anything "bad" about Clinton, et al. I haven't had the time or energy, to be honest, to see if it's simply her, or "liberal-wide," but I suspect, it's not just her. Can you IMAGINE the hue and cry, if CNN discovered that they were doing that on behalf of conservatives? How they'd SCREAM about it? But this--well, this is for the GOOD, right, because it will help get her elected? Honestly...this country is going right to the crapper, in terms of intellectual rigor. Is there some reason that the liberal machine thinks that allowing people to see EVERYTHING, and decide for themselves, is a PROBLEM???? What--they're feeding confirmation bias? Moses on a Pony. And, yes: I'd bitch about this if it was in the other direction, too. It ought NOT be providing 1984-like services, for the formation of public opinion. Anyone here feel all fuzzy about Google--now that you know it's essentially a propagandistic search provider, upon which you may not rely for the facts?

AND, really, about the shows--enough of that crap. It's like watching the infamous "airline versions" of movies. You end up with unresolved plot points, irrational dialogue, etc. PBS wants my dough? Then they need to step up and earn it a bit more than they are now. I waited a year+ just so I could watch Downton Abby without their "HELP" in watching it.

Again, if someone gets all bent, just let me know, and I'll delete the spoilers. In fairness, you've been warned.

</grumble>

Hitch

[grumbles]

Once upon a time, back in the mid 80's, pay TV finally came to Canada. I signed up for "C" Channel which was supposed to be a more "artsy" channel as opposed to the two movie channels that were also offered. Well, artsy it was with things like Twyla Tharpe's "Hot Gossip" but also other offerings like "Last Tango in Paris" which was a first to see that on TV. One of the shows was an one act play, "Rattlesnake in a Cooler". This is an angry hour long diatribe of rage and anger. The language would strip wall paper from the walls. Again, like nothing I had ever seen on TV. A few years later, I saw that the same program was being shown on my local Buffalo PBS station. What I saw was the original video with about an hour of bleeping for a soundtrack. I don't know why they even bothered. I generally skip dram presentations of PBS and wait for the shows to become available on DVD and then get them from the library or these days, grab them off the net from an international source. US broadcast television is a joke. It truly is Minnow's "vast wasteland".

In the early 70's, an small independent station, CITY went on the air in Toronto. It was located at the top of UHF band with their antenna on top of their two storey building in midtown Toronto. If you were lucky, weather conditions were just right and you held the antenna perfectly, you might be able to pick them up as far away as the city limits. To establish themselves as new and important and with the assumption that any children would not be watching after midnight, the launched "The Baby Blue Movie". These were the very softest of soft core movies and did they ever get an audience! If they had tried that in the US they would have all been in jail. Here, the worst that happened was most of the movies were panned as being really boring. There was an upsurge in the purchase of extreme fringe antennas in Buffalo and Niagara Falls, NY though. When CITY moved their transmitter to the CN Tower and their signal became available through Southern Ontario they dropped the Blue Movie series. Our TV programming has become almost as boring as the US now but we still broadcast things late at night that would get the station shut down in the US.

[FizzyWater]

Originally Posted by grumbles

To establish themselves as new and important and with the assumption that any children would not be watching after midnight, the launched "The Baby Blue Movie". These were the very softest of soft core movies and did they ever get an audience!

I remember when we first got cable when I was a teenager (and that was probably late 70s, because I remember it being about the time I went away to college) - one of the pay stations played soft-core porn late at night. I want to say Cinemax, but I really couldn't say for sure. It was pretty awful - we all had a good laugh and that was the end of it.

[grumbles]

I should add, since this was originally about stripping excess carriage returns, I have a couple of command line programs that take text file and unwraps them. The Gutenberg text files for instance, where each line is truncated at about eighty characters. It requires that each paragraph is separated by a blank line. The first program just unwraps from standard input to standard output. The second program does the same thing but wraps the text into a very simple xhtml file. Quotes, single and double, angle brackets and ampersands are converted to entities. I use this as the first stage when converting from text files.

These are very simple programs and are written in Pascal and are compiles with the Free Pascal Compiler. They run on any system that FPC is available for. I use them, and other utilities, on both Windows (XP, 7 and 8.1) as well as Linux.

If anyone is interested, I can post the source. I also have utilities to merge xhtml files or split them on headings.

[JSWolf]

Originally Posted by Hitch

Now, having said that, yes, I do know that the UK systems and ours, for broadcast, are different, and it's highly likely that it sounds just spiffy there. Well, kids, this ain't the 1950's, and any moron can remix something today, for an international audience. I mean, hell, PBS feels justified in "cutting out" the BAD WORDS (Like, ASS. Really???), and editing it for "time," so that we can scrunch our foreheads and wonder about some plot point that was omitted...and they can't remix the bloody thing, to make the music not absurdly over-loud?

UK TV can be annoying (sound wise) because you get the voices rather soft and have to up the volume and then the music and/or sound effects can be rather loud. US TV has the volume sorted better.

[AlexBell]

Originally Posted by grumbles

I should add, since this was originally about stripping excess carriage returns, I have a couple of command line programs that take text file and unwraps them. The Gutenberg text files for instance, where each line is truncated at about eighty characters. It requires that each paragraph is separated by a blank line. The first program just unwraps from standard input to standard output. The second program does the same thing but wraps the text into a very simple xhtml file. Quotes, single and double, angle brackets and ampersands are converted to entities. I use this as the first stage when converting from text files.

These are very simple programs and are written in Pascal and are compiles with the Free Pascal Compiler. They run on any system that FPC is available for. I use them, and other utilities, on both Windows (XP, 7 and 8.1) as well as Linux.

If anyone is interested, I can post the source. I also have utilities to merge xhtml files or split them on headings.

Thanks. I'd certainly be interested in looking at your programs. I have Windows 10, though I don't suppose that makes much difference.