由Android 65K方法数限制引发的思考
ÔÎijö´¦£º½Ü·ç¾Ó¡£
Ç°ÑÔ
ûÏëµ½£¬65536ÕæµÄºÜС¡£
Unable to execute dex: method ID not in \[0, 0xffff\]: 65536
PS:±¾ÎÄÖ»ÊÇ´¿Ì½Ë÷Ò»ÏÂÕâ¸ö65KµÄÀ´Ô´£¬½ö´Ë¶øÒÑ¡£
µ½µ×ÊÇ65k»¹ÊÇ64k?
¶¼Ã»´í£¬Í¬Ò»¸öÎÊÌ⣬²»Í¬µÄ˵·¨¶øÒÑ¡£
65536°´1000ËãµÄ»°£¬ÊÇ65k ~ 65 1000;
65536°´1024ËãµÄ»°£¬ÊÇ64k = 64 1024¡£
ÖصãÊÇ65536=2^16£¬Çë´ó¼Ò¼ÇסÕâ¸öÊý×Ö¡£
ʱ¼äµã
´Ó´ó¼ÒµÄ¾ÀúºÍÕâƪÎÄÕ£º
http://source.android.com/devices/tech/dalvik/dalvik-bytecode.html
À´¿´£¬Õâ¸ö´íÎóÊÇ·¢ÉúÔÚ¹¹½¨Ê±ÆÚ¡£
65536ÊÇÔõôËã³öÀ´µÄ£¿
65536ÍøÉÏÖÚ˵·×ç¡£¬ÓжԵģ¬Óв»È«¶ÔµÄ£¬Ò²ÓдíµÄ¡£
ÏÂÃ潫¸ú×Ù×îеÄAOSPÔ´ÂëÀ´Ë³ÌÙÃþ¹Ï£¬µ«ÊÇ̽Ë÷ÎÊÌâ±ØÈ»ÓØ»ØÈßÓ࣬½ö×÷¼Ç¼£¬¶ÁÕß¿ÉÖ±½ÓÌø¹ý¿´½á¹û¡£
1. Ê×ÏÈ£¬²éÕÒDexµÄ½á¹¹¶¨Òå¡£
/*
* Direct-mapped "header_item" struct.
*/
struct DexHeader {
u1 magic\[8\];
u4 checksum;
u1 signature\[kSHA1DigestLen\];
u4 fileSize;
u4 headerSize;
u4 endianTag;
u4 linkSize;
u4 linkOff;
u4 mapOff;
u4 stringIdsSize;
u4 stringIdsOff;
u4 typeIdsSize;
u4 typeIdsOff;
u4 protoIdsSize;
u4 protoIdsOff;
u4 fieldIdsSize;
u4 fieldIdsOff;
u4 methodIdsSize; // ÕâÀï´æ·ÅÁË·½·¨×Ö¶ÎË÷ÒýµÄ´óС£¬methodIdsSizeµÄÀàÐÍΪu4
u4 methodIdsOff;
u4 classDefsSize;
u4 classDefsOff;
u4 dataSize;
u4 dataOff;
};
u4µÄÀàÐͶ¨ÒåÈçÏ£º
/*
* These match the definitions in the VM specification.
*/
typedef uint8_t u1;
typedef uint16_t u2;
typedef uint32_t u4;
typedef uint64_t u8;
typedef int8_t s1;
typedef int16_t s2;
typedef int32_t s4;
typedef int64_t s8;
½øÒ»²½ÍƳö£¬methodIdsSizeµÄÀàÐÍÊÇuint32_t£¬µ«ËüµÄÏÞÖÆΪ2^32 = 65536 * 65536£¬±È65536´óµÄ¶à¡£
ËùÒÔ£¬65k²»ÊÇdexÎļþ½á¹¹±¾ÉíÏÞÖÆÔì³ÉµÄ¡£
PS£ºDexÎļþÖд洢·½·¨IDÓõIJ¢²»ÊÇshortÀàÐÍ£¬ÎÞÂÛ×îеÄDexFile.hж¨ÒåµÄu4ÊÇuint32_t£¬»¹ÊÇÀÏ°æ±¾DexFileÒýÓõÄvm/Common.hÀﶨÒåµÄu4ÊÇuint32»òÕßunsigned int£¬¶¼²»ÊÇshortÀàÐÍ£¬ÌØ´Ë˵Ã÷¡£
2. DexOptÓÅ»¯Ôì³É£¿
Õâ¸ö˵·¨Ô´×Ô£º
µ±AndroidϵͳÆô¶¯Ò»¸öÓ¦ÓõÄʱºò£¬ÓÐÒ»²½ÊǶÔDex½øÐÐÓÅ»¯£¬Õâ¸ö¹ý³ÌÓÐÒ»¸öרÃŵŤ¾ßÀ´´¦Àí£¬½ÐDexOpt¡£DexOptµÄÖ´Ðйý³ÌÊÇÔÚµÚÒ»´Î¼ÓÔØDexÎļþµÄʱºòÖ´Ðеġ£Õâ¸ö¹ý³Ì»áÉú³ÉÒ»¸öODEXÎļþ£¬¼´Optimised Dex¡£Ö´ÐÐODexµÄЧÂÊ»á±ÈÖ±½ÓÖ´ÐÐDexÎļþµÄЧÂÊÒª¸ßºÜ¶à¡£µ«ÊÇÔÚÔçÆÚµÄAndroidϵͳÖУ¬DexOptÓÐÒ»¸öÎÊÌ⣬Ҳ¾ÍÊÇÕâƪÎÄÕÂÏëҪ˵Ã÷²¢½â¾öµÄÎÊÌâ¡£DexOpt»á°Ñÿһ¸öÀàµÄ·½·¨id¼ìË÷ÆðÀ´£¬´æÔÚÒ»¸öÁ´±í½á¹¹ÀïÃæ¡£µ«ÊÇÕâ¸öÁ´±íµÄ³¤¶ÈÊÇÓÃÒ»¸öshortÀàÐÍÀ´±£´æµÄ£¬µ¼ÖÂÁË·½·¨idµÄÊýÄ¿²»Äܹ»³¬¹ý65536¸ö¡£µ±Ò»¸öÏîÄ¿×ã¹»´óµÄʱºò£¬ÏÔÈ»Õâ¸ö·½·¨ÊýµÄÉÏÏÞÊDz»¹»µÄ¡£¾¡¹ÜÔÚа汾µÄAndroidϵͳÖУ¬DexOptÐÞ¸´ÁËÕâ¸öÎÊÌ⣬µ«ÊÇÎÒÃÇÈÔÈ»ÐèÒª¶ÔÀÏϵͳ×ö¼æÈÝ¡£
¼øÓÚÎÒÄÜÁ¦ÓÐÏÞ£¬Ã»ÓÐÕÒµ½Õâ¿éÂß¼¶ÔÓ¦µÄ´úÂë¡£
µ«ÎÒÓиöÒÉÎÊ£¬Õâ¸öÏÞÖÆÊÇÔÚAndroidÆô¶¯Ò»¸öÓ¦ÓõÄʱºò·¢ÉúµÄ£¬µ«´ÓÇ°ÃæµÄ¡°Ê±¼äµã¡±Õ½ڣ¬65kÎÊÌâÊÇÔÚ¹¹½¨µÄʱºò¾Í·¢ÉúÁË£¬»¹Ã»µ½Æô¶¯»òÕßÔËÐÐÕâÒ»²½¡£
ÎÒ²»¸Ò·ñ¶¨ÕâÖÖ˵·¨£¬µ«ËµÃ÷65kÖÁÉÙ»¹ÓÐÆäËûµØ·½ÏÞÖÆ¡£
3. DexMergerµÄ¼ì²â
Ö»ÄÜÔÚdalvikĿ¼ÏÂËÑË÷¹Ø¼ü×Ö¡±methid ID not in¡±£¬ÔÚDexMerggerÀïÕÒµ½ÁËÅ׳öÒì³£µÄµØ·½£º
/**
* Combine two dex files into one.
*/
public final class DexMerger {
private void mergeMethodIds() {
new IdMerger<MethodId>(idsDefsOut) {
@Override TableOfContents.Section getSection(TableOfContents tableOfContents) {
return tableOfContents.methodIds;
}
@Override MethodId read(Dex.Section in, IndexMap indexMap, int index) {
return indexMap.adjust(in.readMethodId());
}
@Override void updateIndex(int offset, IndexMap indexMap, int oldIndex, int newIndex) {
if (newIndex < 0 || newIndex > 0xffff) {
throw new DexIndexOverflowException(
"method ID not in \[0, 0xffff\]: " + newIndex);
}
indexMap.methodIds\[oldIndex\] = (short) newIndex;
}
@Override void write(MethodId methodId) {
methodId.writeTo(idsDefsOut);
}
}.mergeSorted();
}
}
ÕâÀﶨÒåÁËindexMapµÄmethodIdsµÄµ¥ÏîֵҪǿתshort£¬ËùÒÔÔÚ´æ·Å֮ǰcheckһϷ¶Î§ÊDz»ÊÇ0 ~ 0xffff¡£
ÎÒÃÇ¿´¿´IndexMapµÄ¶¨Ò壺
/**
* Maps the index offsets from one dex file to those in another. For example, if
* you have string #5 in the old dex file, its position in the new dex file is
* {@code strings\[5\]}.
*/
public final class IndexMap {
private final Dex target;
public final int\[\] stringIds;
public final short\[\] typeIds;
public final short\[\] protoIds;
public final short\[\] fieldIds;
public final short\[\] methodIds;
// ... ...
}
¿´ÉÏÈ¥ÊǶÔÁË£¬¿ÉÊÇÕâ¸öDexMergerÊǺϲ¢Á½¸ödexµÄ£¬Ä¬ÈÏÇé¿öÏÂÎÒÃÇÖ»ÓÐÒ»¸ödexµÄ£¬ÄÇôÕâ¸ö65kÊÇÄÄÀïÏÞÖƵÄÄØ£¿Ôٲ飡
4. »Ø¹éDexFile
»ù±¾ÉÏÇ°Ãæ»ù±¾ÊÇÒ»¸öÃþ×Åʯͷ¹ýºÓ¡¢·´¸´ÑéÖ¤ÍøÂç˵·¨µÄÒ»¸ö¹ý³Ì£¬ËäÈ»»ØÏëÆðÀ´ÉµÉµµÄ£¬µ«ÊÇÕâÖּǼ»¹ÊÇÓбØÒªµÄ¡£
Ç°Ãæ¿´µ½DexFileµÄ´æ·Å·½·¨Êý´óСµÄÀàÐÍÊÇuint32£¬µ«ÊǸù¾ÝºóÃæµÄÅжϣ¬ÎÒÃÇÈ·¶¨ÊÇ´ò°üµÄ¹ý³ÌÖвúÉúÁË65kÎÊÌ⣬ËùÒÔÎÒÃǵûعýÍ·ÀÏÀÏʵʵÑо¿Ò»ÏÂdxµÄ´ò°üÁ÷³Ì¡£
¡ ´Ë´¦Ê¡ÂÔ·ÖÎöÁ÷³Ì5000×Ö ¡
OK£¬ÎÒ°Ñdx´ò°üÉæ¼°µ½Á÷³Ì¼Ç¼ÏÂÀ´£º
// Ô´ÂëĿ¼£ºdalvik/dx
// Main.java
-> main() -> run() -> runMonoDex()(»òÕßrunMultiDex()) -> writeDex()
// DexFile
-> toDex() -> toDex0()
// MethodIdsSection extends MemberIdsSection extends UniformItemSection extends Section
-> prepare() -> prepare0() -> orderItems() -> getTooManyMembersMessage()
// Main.java
-> getTooManyIdsErrorMessage()
×îÖÕºüÀêµÄβ°ÍÊÇÔÚMemberIdsSection©³öÀ´ÁË£º
package com.android.dx.dex.file;
import com.android.dex.DexException;
import com.android.dex.DexFormat;
import com.android.dex.DexIndexOverflowException;
import com.android.dx.command.dexer.Main;
import java.util.Formatter;
import java.util.Map;
import java.util.TreeMap;
import java.util.concurrent.atomic.AtomicInteger;
/**
* Member (field or method) refs list section of a {@code .dex} file.
*/
public abstract class MemberIdsSection extends UniformItemSection {
/**
* Constructs an instance. The file offset is initially unknown.
*
* @param name {@code null-ok;} the name of this instance, for annotation
* purposes
* @param file {@code non-null;} file that this instance is part of
*/
public MemberIdsSection(String name, DexFile file) {
super(name, file, 4);
}
/** {@inheritDoc} */
@Override
protected void orderItems() {
int idx = 0;
if (items().size() > DexFormat.MAX_MEMBER_IDX + 1) {
throw new DexIndexOverflowException(getTooManyMembersMessage());
}
for (Object i : items()) {
((MemberIdItem) i).setIndex(idx);
idx++;
}
}
private String getTooManyMembersMessage() {
Map<String, AtomicInteger> membersByPackage = new TreeMap<String, AtomicInteger>();
for (Object member : items()) {
String packageName = ((MemberIdItem) member).getDefiningClass().getPackageName();
AtomicInteger count = membersByPackage.get(packageName);
if (count == null) {
count = new AtomicInteger();
membersByPackage.put(packageName, count);
}
count.incrementAndGet();
}
Formatter formatter = new Formatter();
try {
String memberType = this instanceof MethodIdsSection ? "method" : "field";
formatter.format("Too many %s references: %d; max is %d.%n" +
Main.getTooManyIdsErrorMessage() + "%n" +
"References by package:",
memberType, items().size(), DexFormat.MAX_MEMBER_IDX + 1);
for (Map.Entry<String, AtomicInteger> entry : membersByPackage.entrySet()) {
formatter.format("%n%6d %s", entry.getValue().get(), entry.getKey());
}
return formatter.toString();
} finally {
formatter.close();
}
}
}
ÀïÃæÓÐÒ»¶Î£º
// Èç¹û·½·¨Êý´óÓÚ0xffff¾ÍÌáʾ65k´íÎó
if (items().size() > DexFormat.MAX_MEMBER_IDX + 1) {
throw new DexIndexOverflowException(getTooManyMembersMessage());
}
// Õâ¸öDexFormat.MAX_MEMBER_IDX¾ÍÊÇ0xFFFF
/**
* Maximum addressable field or method index.
* The largest addressable member is 0xffff, in the "instruction formats" spec as field@CCCC or
* meth@CCCC.
*/
public static final int MAX_MEMBER_IDX = 0xFFFF;
ÖÁ´Ë£¬ÕæÏà´ó°×£¡
5. ¸ù±¾ÔÒò
Ϊʲô¶¨ÒåDexFormat.MAX_MEMBER_IDXΪ0xFFFF?
ËäÈ»ÎÒÃÇÕÒµ½ÁË65k±¨´íµÄµØ·½£¬µ«ÊÇΪʲô³ÌÐòÖз½·¨Êý³¬¹ý0xFFFF¾ÍÒª±¨´íÄØ£¿
ͨ¹ýËÑË÷¡±instruction formats¡±, ÎÒ×îÖղ鵽ÁËDalvik VM Bytecode£¬ÕÒµ½×îеĹٷ½ËµÃ÷£º
https://source.android.com/devices/tech/dalvik/dalvik-bytecode.html
ÀïÃæ˵Ã÷ÁËÉÏÃæµÄ@CCCCµÄ·¶Î§±ØÐëÔÚ0¡«65535Ö®¼ä£¬ÕâÊÇdalvik bytecodeµÄÏÞÖÆ¡£
ËùÒÔ£¬65536ÊÇbytecodeµÄ16λÏÞÖÆËã³öÀ´µÄ£º2^16¡£
PS£ºÒÔÉÏ·ÖÎöµÃµ½ÈºÀïºÜ¶àÅóÓѵÄÌÖÂۺͰïæ¡£
6. »Ø¹Ë
ÎÒºÃÏñÃ÷°×ÁËʲô£º
-
65kÎÊÌâÊÇdx´ò°üµ¥¸öDexʱ±¨µÄ´í£¬ËùÒÔÖ»ÒªÓÃdx´ò°üµ¥¸ödex¾Í¿ÉÄÜÓÐÕâ¸öÎÊÌâ¡£
-
²»½ö·½·¨Êý£¬×Ö¶ÎÊýÒ²ÓÐ65kÎÊÌâ¡£
-
Ä¿Ç°À´Ëµ£¬65kÎÊÌâºÍϵͳÎ޹ء£
-
Ä¿Ç°À´Ëµ£¬65kÎÊÌâºÍartÎ޹ء£
-
¼´Ê¹·Ö°üMultiDex£¬µ±Ö÷DexµÄ·½·¨Êý³¬¹ý65kÒÀÈ»»á±¨´í¡£
-
MultiDex·½°¸²»ÊÇ´Ó¸ù±¾ÉϽâ¾öÁË65kÎÊÌ⣬µ«ÊÇ´ó´ó»º½âÉõÖÁ˵»ù±¾½â¾öÁË65kÎÊÌâ¡£
еÄJackÄÜ·ñ½â¾ö65kÎÊÌ⣿
¾Ý˵JackµÄ·½Ê½°Ñclass´ò°ü³É.jackÎļþ¡£
ËùÒÔÎÒÈÏΪ£¬Jack¾ß±¸½â¾ö65kÎÊÌâµÄÌõ¼þ£º
-
´ò°ü£ºÐµÄjackÎļþ¿Ï¶¨ÊÇÅ×ÆúÁËdalvikµÄ¼æÈÝÐÔ£¬ÕâÒ²×¢¶¨ÔÛÃÇÕâÁ½Äê¿ÉÄÜ»¹Óò»ÁË¡£
-
ÐéÄâ»ú£ºÍêÈ«²ÉÓÃеÄARTÐéÄâ»ú£¬°Ñclassת»¯³É±¾µØ»úÆ÷Â룬¾ÍÄܱܿªdalvik bytecodeµÄ16λÏÞÖÆ¡£
-
ÉÏÃæÁ½ÌõÊôÓÚ·Ï»°£¬Ëµ°×ÁË£¬ÍêÈ«²»ÓÃdalvikÐéÄâ»úÁË£¬Í¬Ê±Ò²¾ÍÍêÈ«²»ÓÃdxÁË£¬Èç´Ë£¬µ±È»¾Í²»´æÔÚ65kÎÊÌâÁË¡£
ÒÔÉÏ´¿ÊôÎÒ¸öÈËÍƲ⣬һÇÐÒÔ¿Æѧ·ÖÎöΪ׼¡£
С½á
Á˽â65kÎÊÌâ²»»á¶Ô¹¤×÷ÓÐʲô°ïÖú£¬¹¤×÷żÓö£¬ÂÔ×öÊáÀí£¬×öÒ»×ܽᣡ