Aleksandras Kostarevas
350b8e8fcf
Add bad word filtering and blacklisting
2024-03-13 13:31:51 -05:00
Aleksandras Kostarevas
9fed68c03a
Fix segfault when no results / only 1 result
2024-03-07 14:56:21 +02:00
Aleksandras Kostarevas
69649256f8
Fix comma key to open settings menu
2024-03-05 15:54:18 +02:00
Aleksandras Kostarevas
c57a3d83af
Add personal dictionary glossary for voice input and keyboard
2024-03-05 15:24:30 +02:00
Aleksandras Kostarevas
42ac255a81
Sync whisper.cpp changes from voice input
2024-03-05 11:06:24 +02:00
Aleksandras Kostarevas
e4d41567b0
Update hyperparameter
2024-02-20 21:04:07 +02:00
Aleksandras Kostarevas
6453c15a21
Merge branch 'lm-2-finetuning-whisperggml' into 'model-metadata'
...
Add autocorrect threshold to model-metadata branch
See merge request alex/latinime!6
2024-02-03 15:18:27 +00:00
Aleksandras Kostarevas
c7113297fb
Add radio selection for threshold
2024-02-01 21:55:56 +02:00
Aleksandras Kostarevas
a111164bb8
Improve algorithm in a few ways:
...
* If the first letter is capital, only capitalized first tokens will be sampled. If the whole text is capitalized, then only fully capital tokens will be sampled for the whole word
* If a word is an exact match, it gets boosted relative to others
* Probability threshold for autocorrect is now 18.0
* Add "clueless" threshold, if it's less than 1.3 then just show the user's typed word in the middle instead.
2024-01-30 20:30:44 +02:00
Aleksandras Kostarevas
f888ba3353
Fix finetuning, add a finetuning screen, handle errors during importing model, update metadata format, add model exporting
2024-01-30 17:14:02 +02:00
Aleksandras Kostarevas
5bf4492634
Fix embedded tokenizer loading, implement new model management methods, implement model info loading, model importing
2024-01-28 22:40:39 +02:00
Aleksandras Kostarevas
0021b6aa04
Model metadata and manager component
2024-01-24 01:03:16 +02:00
Aleksandras Kostarevas
7aea41eede
Add decomposeTapPosition fallback for nonintersecting cases
2024-01-22 08:21:27 +02:00
Aleksandras Kostarevas
5e0722c984
Fix issue with apostrophe token being banned
2024-01-22 08:20:55 +02:00
Aleksandras Kostarevas
dbad61d2e6
Fix non-English dictionary prediction
2024-01-16 21:02:55 +02:00
Aleksandras Kostarevas
55d5959f54
Skip non-alphabetic characters during mixing
2024-01-09 18:25:14 +02:00
Aleksandras Kostarevas
ebb70b9c12
Fix build, disable gesture input pending model update
2023-12-19 20:28:58 +02:00
Aleksandras Kostarevas
4e9e86d871
Implement multimodal position encoder
2023-12-19 20:02:20 +02:00
Aleksandras Kostarevas
314cf8c84c
Type out whisper.cpp result
2023-12-05 18:06:12 +00:00
Aleksandras Kostarevas
7075c22179
Add key embedding mixing
2023-12-04 20:09:51 +00:00
Aleksandras Kostarevas
4f15ff4a73
Add experimental swipe typing
2023-11-28 17:01:58 +00:00
Aleksandras Kostarevas
854e1295cc
Fix problem with n_tokens==0
2023-11-28 16:20:33 +00:00
abb128
ca9c9d5a9a
ggml backend v2
2023-11-25 09:39:04 +02:00
abb128
f31db527d6
Add whisper.cpp
2023-11-25 09:13:50 +02:00
Aleksandras Kostarevas
001f1eb442
Disable debug flag
2023-11-21 20:33:21 +02:00
Aleksandras Kostarevas
cb2edca601
Update training hyperparameters
2023-11-21 17:07:43 +02:00
Aleksandras Kostarevas
14fcb55565
Save LoRA-merged model after training
2023-11-14 20:40:00 +02:00
Aleksandras Kostarevas
2409eecef5
Loss/progress training callbacks
2023-11-14 18:11:00 +02:00
Aleksandras Kostarevas
b53a46b18d
Move training to CoroutineWorker
2023-11-14 17:23:08 +02:00
Aleksandras Kostarevas
38b06d7909
History logging and training based on log
2023-11-14 11:43:36 +02:00
Aleksandras Kostarevas
0e0876f06c
Revise training
2023-11-13 16:42:01 +02:00
Aleksandras Kostarevas
ee8a81f12c
Initial fine-tuning
2023-11-07 16:48:48 +02:00
Aleksandras Kostarevas
5778cd15a0
Update ggml and llama.cpp
2023-11-06 13:41:25 +02:00
Aleksandras Kostarevas
7c4531e32d
Fix crashes related to too large context
2023-10-16 18:24:00 +03:00
Aleksandras Kostarevas
c73fe16ddc
Add q8 model
2023-10-16 17:31:43 +03:00
Aleksandras Kostarevas
92480fd460
Adjust space probability and mustNotAutocorrect
2023-10-13 18:44:38 +03:00
Aleksandras Kostarevas
c34a411989
Fix infinite prediction loop
2023-10-13 18:34:49 +03:00
Aleksandras Kostarevas
b8539ce88a
Initial batched inference using llama_batch
2023-10-10 22:34:04 +03:00
Aleksandras Kostarevas
6d17f00296
Update llama.cpp
2023-10-10 22:32:25 +03:00
Aleksandras Kostarevas
16fdb3629d
Add LanguageModel class
2023-09-28 19:42:29 +03:00
Aleksandras Kostarevas
ea0af67ecc
Add ggml, sentencepiece and dependencies
2023-09-28 19:41:05 +03:00
abb128
434f8b6b27
Initial working build of fork
2023-07-06 21:57:49 +03:00
Jing Mike
03eef94a8d
Remove unused variables
...
Since some variables with module LatinIME are defined but not used,
when compiled with build combination "sdk_pc_x86_64-userdebug" and
build command "mmm packages/inputmethods/LatinIME", the following
code lines will be reported that "variable 'XXX' set but not used".
(should be similar for all the other build combinations)
Repeated 10 times for each:
terminal_position_lookup_table.cpp:74:9 removedEntryCount
terminal_position_lookup_table.cpp:85:9 removedEntryCount
proximity_info_state_utils.cpp:493:9 tempTime
trie_map.cpp:56:9 unusedRegionSize
suggestion_results.cpp💯 9 index
Repeated 80+ times:
proximity_info_utils.h:75:25 proximityChar
With this patch we are removing some of the unused variables and
putting the C++ 17 attribute [[maybe_unused]] to the others which
are used for logging. Then all the related build warnings have been
eliminated.
Test: mmm packages/inputmethods/LatinIME, presubmit check.
Change-Id: Ia66766322d6ae8a010b1cb55cc22993fbc6d012c
Signed-off-by: Jing Mike <jingyangliu@eswincomputing.com>
2023-03-19 10:00:01 +00:00
Bob Badour
f3d9532a32
[LSC] Add LOCAL_LICENSE_KINDS to packages/inputmethods/LatinIME
...
Added SPDX-license-identifier-Apache-2.0 to:
Android.bp
common/Android.bp
java/Android.bp
native/dicttoolkit/Android.bp
native/jni/Android.bp
tests/Android.bp
tools/EditTextVariations/Android.bp
tools/dicttool/Android.bp
tools/make-keyboard-text/Android.bp
Bug: 68860345
Bug: 151177513
Bug: 151953481
Test: m all
Exempt-From-Owner-Approval: janitorial work
Change-Id: I440008bffac5c97a2497970af377a9d03262b6d8
2021-02-17 09:46:27 -08:00
Treehugger Robot
e04480b68b
Merge "Mark liblatinime_unittests as unit_test:true to run in presubmit in CI"
2021-02-17 10:24:20 +00:00
Julien Desprez
b19114e93b
Mark liblatinime_unittests as unit_test:true to run in presubmit in CI
...
This will ensure that the target stays green.
Note: we will clean up the explicit unit_tests:true once cc_test default is changed.
Change-Id: I5153ca918e94b370b1f08289118877d209385ba2
Test: presubmit
Bug: 180014759
2021-02-16 18:31:04 +00:00
Adrian Roos
8b3d8f48e1
Fix broken liblatinime_unittests
...
Change-Id: I95021965ecc2699fbce01ed0552dd28fed7f8fe4
Merged-In: I95021965ecc2699fbce01ed0552dd28fed7f8fe4
Fixes: 180014759
Test: atest liblatinime_unittests --host
2021-02-16 17:49:08 +00:00
Seigo Nonaka
0c549e7424
Update language to comply with Android’s inclusive language guidance
...
See https://source.android.com/setup/contribute/respectful-code for reference
Bug: 161896447
Test: tapas LatinIME && m
Change-Id: Ic06f948c023a091e1532db826a78dd01031d7205
2020-07-31 11:28:59 -07:00
Jeff Sharkey
1d9bf42009
Update language to comply with Android's inclusive language guidance
...
See https://source.android.com/setup/contribute/respectful-code for reference
Bug: 161896447
Change-Id: I171bc299e660f96f20041a67a3052bd8a2ed781c
2020-07-31 09:33:35 -06:00
Orion Hodson
54c4f2afb5
Add explicit dependencies on jni_headers
...
Preparation for removing implicit include paths for jni.h from soong.
Bug: 152482542
Test: lunch aosp_x86_64 && m checkbuild
Change-Id: Ib36e679819a3309ccbf12fd592309b16c8bbd6c8
2020-04-13 17:15:59 +01:00
