This reverts commit bdbe279cec
Other half of this change couldn't be committed because of a inter-project directory migration. Reverting until things settle down.
Change-Id: I3862fc8485dfca6d507f620e905cfa583aa0e39a
Log statements are now recorded with a timestamp. This is important for filtering out statements
not part of words that are sampled when spaces are inserted automatically.
multi-project commit with Change-Id: I68221a2528045d25632aef4bb716f92a4f4a56a4
Change-Id: I9f49c36391e1e1773b6d4876d7387c893024f87b
Previously, a logunit was considered a word only if it was all letters. This is important for
tracking bigrams correctly.
Now, a logunit must have only at least one letter. The dictionary check is still performed,
and punctuation, etc. still comes in as separate LogUnits. But a word can contain a space,
which helps set up for logging words where spaces are inserted automatically, and other situations
in which text is committed with an additional space tacked onto the end.
Change-Id: Ia74094a99058890d20a9cdadf2d0989841a79a41
* commit 'faee53c0880c637acd9bf23b4a7aadeeb4a50085':
Revert "Correctly add double quote to the space strippers"
Revert "Only add spaces automatically when appropriate."
This brings the changes of I837831b1 to MR1.1
>>> dictionaries/ru_wordlist.combined.gz
>>> java/res/raw/main_ru.dict
Header :
date : 1353500945 <=> 1353567943
MULTIPLE_WORDS_DEMOTION_RATE : null <=> 0
Body :
No differences
Bug: 7540132
Change-Id: I81f281fda881ad1252cdf6cf2d0947a12a2dac27
This brings the dictionaries to the same level as
master. The change in master reflected by this
change is I94fe7f8f.
- XML de dict :
Update header only.
- XML en dict :
Add the not_a_word flag to all whitelist entries.
Demote "il:92" and "shel:52" to freq 0.
- XML en_GB dict :
Add the not_a_word flag to all whitelist entries.
Add "milk:122" and "practice:143"
- XML fr dict :
Add missing words : "Nostradamus:40" "défendais:30"
"gmail:50" "générale:140" "hm:0" "hmm:0" "y'en:130"
"l'apocalypse:31" "m'épuise:30" "recontacter:80"
"t'annonce:30"
Set freq of non-word shortcuts for digraphs to 1 instead
of 0, allowing to gesture them.
- XML ru dict :
Remove a lot of two-character non-words.
- Binary de dict :
Remove the obsolete "options" header, and add the "dictionary"
header.
- Binary en dict :
Flag "hoe" "hoes" "il" "shel" as non-words.
Also drop freq of "il" and "shel" to 0
Add the "locale" header that was missing.
- Binary es dict :
Add the "dictionary" header.
- Binary fr dict :
Add the same words as above. Non-word shortcuts were already
set to 1.
- Binary it dict :
Add a "dictionary" header. Also change freq of
"Šarapova" from 50 to 37; not sure why it was 50.
- Binary pt_BR dict :
Add a "dictionary" header.
- Binary ru dict :
Add a "dictionary" header and remove the same words as above.
Bug: 7458565
Change-Id: I22dd176ed2df99b847047f2d1045f6d861ccb8b0
- Combined de dict :
Remove digraph shortcuts that were in by mistake.
- Combined en dict :
Set freq of "baton" "batons" "mace" "puff"
"puffs" and "tasers" to zero. They are offensive
in en_GB.
- Combined en_GB dict :
Change freq of "il" to 0 and flag it "not a word". Still
in the dict as a whitelist entry for "I'll"; for some
reason it had freq 99.
Add "milk:122" and "practice:143"
- Combined fr dict :
Add missing words : "Nostradamus:40" "défendais:30"
"gmail:50" "générale:140" "hm:0" "hmm:0" "y'en:130"
"l'apocalypse:31" "m'épuise:30" "recontacter:80"
"t'annonce:30"
Set freq of non-word shortcuts for digraphs to 1 instead
of 0, allowing to gesture them.
- Combined ru dict :
Remove a lot of two-character non-words.
- Binary de dict :
Remove the obsolete "options" header, and add the "dictionary"
header.
- Binary en dict :
Flag "hoe" "hoes" "il" "shel" as non-words.
Also drop freq of "il" and "shel" to 0
Add the "locale" header that was missing.
- Binary es dict :
Add the "dictionary" header.
- Binary fr dict :
Add the same words as above. Non-word shortcuts were already
set to 1.
- Binary it dict :
Add a "dictionary" header. Also change freq of
"Šarapova" from 50 to 37; not sure why it was 50.
- Binary pt_BR dict :
Add a "dictionary" header.
- Binary ru dict :
Add a "dictionary" header and remove the same words as above.
For all dictionaries : bump the version to 27.
Change-Id: I94fe7f8f42b31fdad223085c00a94115e14d2276
