python - In what 8-bit character set is 0x9d meaningful? -


in 8-bit ascii-like character set english 0x9d meaningful? i'm cleaning old data files, , finding 0x9d in otherwise-ascii text. (no, it's not utf-8.)

it's not valid in windows-1252. python "latin-1" codec translates unicode 0x9d, "operating system command". makes little sense. in unicode box [009d]. (in python, can convert latin-1 without errors being raised, doesn't mean it's meaningful so.)

examples, python-type escapes, messy database i'm cleaning combines text many sources:

guitar pro, jamplay, redbana\\\'s audition,\x9d doppleganger\x99s lounge\x9d or heatwave interactive\x99s platinum life country,\\"  example \\"i\\\'ve seen bull run in pamplona, spain\x9d.\\"  netwise depot  \\"one stop web shop\\"\x9d provides sustainable \\"green\\"\x9d living  looking \\"do me\\"\x9d solution

from context, i'd suspect ™ or ®. 8-bit code had those?

here's wild hypothesis:

some prior (really broken) system working on data attempted write each character utf-8, wrote last byte of each sequence (maybe had weird one-byte-long buffer somewhere). alternatively, in utf-8 in past, viewing in different encoding did search-and-replace remove bytes 0xe2 0x80 because "didn't belong" , didn't realize remaining "special character" wasn't 1 wanted either.

ascii, of course, passed through utf-8 encoding 1 byte long.

the 'right single quotation mark' (u+2019) ’ encoded in utf-8 bytes 0xe2 0x80 0x99. places have \x99s made me go down path, since apostrophe before s translated right curly quotation mark in popular word processing software. if last byte of character saved, you'd have 0x99 there.

the 'right double quotation mark' (u+201d) ” encoded in utf-8 bytes 0xe2 0x80 0x9d. 0x9d have in text @ end of double-quoted string. and, it's right next regular straight " double-quote. wonder if had tried sort of prior clean-up pass on data, , managed put in closing quote, left "weird" 0x9d in there.

as said, it's wild hypothesis, if conglomeration of data variety of old systems, it's hard know may have happened it. last byte of utf-8 closest "normal" english encoding find have reasonable in english text , included bytes looking for.


Comments

Post a Comment

Popular posts from this blog

Is there a better way to structure post methods in Class Based Views -

i'm using class based views in django , never specify url in ajax calls , omit action parameter in form because know these requests go through post method of class based view. notice view - profile page make many requests depending on user updating on page - post method become convoluted , if/else mess example: def post(self, request, *args, **kwargs): # handle user changing available date if request.post.get('availabledate') != none: self.updateavailabledate(request) return jsonresponse({'result':'success'}) if request.post.get('newprojectname') != none: creative_user = creativeuserprofile.objects.get(id = request.user.id) project = project.create_project(creative_user, request.post['newprojectname']) project.save() return jsonresponse({'projectid': project.id }) if request.post.get('projectimage') != none , request.files['file']: proj
Read more

performance - Why is XCHG reg, reg a 3 micro-op instruction on modern Intel architectures? -

i'm doing micro-optimization on performance critical part of code , came across sequence of instructions (in at&t syntax): add %rax, %rbx mov %rdx, %rax mov %rbx, %rdx i thought had use case xchg allow me shave instruction , write: add %rbx, %rax xchg %rax, %rdx however, dimay found agner fog's instruction tables , xchg 3 micro-op instruction 2 cycle latency on sandy bridge, ivy bridge, broadwell, haswell , skylake. 3 whole micro-ops , 2 cycles of latency! 3 micro-ops throws off 4-1-1-1 cadence , 2 cycle latency makes worse original in best case since last 2 instructions in original might execute in parallel. now... cpu might breaking instruction micro-ops equivalent to: mov %rax, %tmp mov %rdx, %rax mov %tmp, %rdx where tmp anonymous internal register , suppose last 2 micro-ops run in parallel latency 2 cycles. given register renaming occurs on these micro-architectures, though, doesn't make sense me done way. why wouldn't register renamer
Read more

jquery - Responsive Navbar with Sub Navbar -

good day, forgive me nav vocabulary not best. if question confusing apologize. i'm working on navigation bar needs centered on page @ 768px , above. items in nav open (or perhaps toggle) subnav lives directly below. these nav should appear centered on page. i have been able work through responsive portion of nav bar, having main items align vertically , sub nav menu of each item roll out below, , gave them indent separation. working well. however, issue not responsive side, behavior @ width greater 768px. happens toggling menu's sub navigation pushes other main items down page below sub nav. here's code: $(document).ready(function () { //stick in fixed 100% height behind navbar don't wrap $('#slide-nav.navbar-inverse').after($('<div class="inverse" id="navbar-height-col"></div>')); $('#slide-nav.navbar-default').after($('<div id="navbar-height-col"></d
Read more